Browse > Article
http://dx.doi.org/10.3837/tiis.2021.10.008

Cascade Network Based Bolt Inspection In High-Speed Train  

Gu, Xiaodong (School of Mathematics and Information Technology, Jiangsu Second Normal University)
Ding, Ji (School of Physics and Electronic Engineering, Jiangsu Second Normal University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.15, no.10, 2021 , pp. 3608-3626 More about this Journal
Abstract
The detection of bolts is an important task in high-speed train inspection systems, and it is frequently performed to ensure the safety of trains. The difficulty of the vision-based bolt inspection system lies in small sample defect detection, which makes the end-to-end network ineffective. In this paper, the problem is resolved in two stages, which includes the detection network and cascaded classification networks. For small bolt detection, all bolts including defective bolts and normal bolts are put together for conducting annotation training, a new loss function and a new boundingbox selection based on the smallest axis-aligned convex set are proposed. These allow YOLOv3 network to obtain the accurate position and bounding box of the various bolts. The average precision has been greatly improved on PASCAL VOC, MS COCO and actual data set. After that, the Siamese network is employed for estimating the status of the bolts. Using the convolutional Siamese network, we are able to get strong results on few-shot classification. Extensive experiments and comparisons on actual data set show that the system outperforms state-of-the-art algorithms in bolt inspection.
Keywords
Few-shot learning; Siamese network; Small object detection; Non-maximum suppression; Convolutional neural networks;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Feifei L, Fergus R, Perona P, et al., "One-shot learning of object categories," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28(4), pp. 594-611, 2006.   DOI
2 Munkhdalai T, Yu H., "Meta networks," in Proc. of international conference on machine learning, pp. 2554-2563, 2017.
3 M. Everingham, L. Van Gool, C. K. I.Williams, J. Winn, and A. Zisserman, "The pascal visual object classes (voc) challenge," International Journal of Computer Vision, vol. 88, no.2, pp. 303-338, 2010.   DOI
4 Chen J, Liu Z, Wang H, et al., "Automatic Defect Detection of Fasteners on the Catenary Support Device Using Deep Convolutional Neural Network," IEEE Transactions on Instrumentation and Measurement, vol. 67(2), pp. 257-269, 2018.   DOI
5 Angadi, S., & Nandyal, S., "Human Identification System Based on Spatial and Temporal Features in the Video Surveillance System," International Journal of Ambient Computing and Intelligence, vol. 11(3), pp. 1-21, 2020.   DOI
6 Aytekin C, Rezaeitabar Y, Dogru S, et al., "Railway Fastener Inspection by Real-Time Machine Vision," systems man and cybernetics, vol. 45(7), pp. 1101-1107, 2015.
7 Dou Y, Huang Y, Li Q, et al., "A fast template matching-based algorithm for railway bolts detection," International Journal of Machine Learning and Cybernetics, vol. 5(6), pp. 835-844, 2014.   DOI
8 Cha Y, You K, Choi W, et al., "Vision-based detection of loosened bolts using the Hough transform and support vector machines," Automation in Construction, vol. 71, pp. 181-188, 2016.   DOI
9 T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick, "Microsoft coco: Common objects in context," in Proc. of European conference on computer vision, Springer, pp. 740-755, 2014.
10 Feng H, Jiang Z, Xie F, et al., "Automatic Fastener Classification and Defect Detection in Vision-Based Railway Inspection Systems," IEEE Transactions on Instrumentation and Measurement, vol. 63(4), pp. 877-888, 2014.   DOI
11 Wang T, Tan N, Zhang C, et al., "A Novel Sparse Representation Based Visual Tracking Method for Dynamic Overhead Cranes: Visual Tracking Method for Dynamic Overhead Cranes," International Journal of Ambient Computing and Intelligence, vol.10, no.4, pp. 45-59, 2019.   DOI
12 Chen, W., Li, Y., & Li, C., "A Visual Detection Method for Foreign Objects in Power Lines Based on Mask R-CNN," International Journal of Ambient Computing and Intelligence, vol. 11(1), pp. 34-47, 2020.   DOI
13 Cha Y, Choi W, Buyukozturk O, et al., "Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks," Computer-aided Civil and Infrastructure Engineering, vol. 32(5), pp. 361-378, 2017.   DOI
14 Marino F, Distante A, Mazzeo P L, et al., "A Real-Time Visual Inspection System for Railway Maintenance: Automatic Hexagonal-Headed Bolts Detection," systems man and cybernetics, vol. 37(3), pp. 418-428, 2007.   DOI
15 Gibert, X.; Patel, V. M. Chellappa, R., "Deep Multitask Learning for Railway Track Inspection," IEEE Transactions on Intelligent Transportation Systems, vol. 18, pp. 153-164, 2017.   DOI
16 Lecun Y, Bottou L, B. Y., et al., "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86(11), pp. 2278-2324, 1998.   DOI
17 Russakovsky O, Deng J, Su H, et al., "ImageNet Large Scale Visual Recognition Challenge," International Journal of Computer Vision, vol. 115(3), pp. 211-252, 2015.   DOI
18 Kisantal M, Wojna Z, Murawski J, et al., "Augmentation for small object detection," arXiv: Computer Vision and Pattern Recognition, pp. 119-133, 2019.
19 Lin T, Dollar P, Girshick R, et al., "Feature Pyramid Networks for Object Detection," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 936-944, 2017.
20 Espada J P, Martinez O S, Garciabustelo B C, et al., "Virtual Objects on the Internet of Things," International Journal of Interactive Multimedia and Artificial Intelligence, vol.1(4), pp. 23-29, 2011.   DOI
21 Krizhevsky A, Sutskever I, Hinton G E, et al., "ImageNet Classification with Deep Convolutional Neural Networks," Communications of the ACM, Vol. 60(6), pp. 84-90, 2017.   DOI
22 Szegedy C, Liu W, Jia Y, et al., "Going deeper with convolutions," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1-9, 2015.
23 Girshick R, Donahue J, Darrell T, et al., "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 580-587, 2014.
24 Girshick R., "Fast R-CNN," in Proc. of international conference on computer vision, pp. 1440-1448, 2015.
25 Redmon J, Farhadi A., "YOLO9000: Better, Faster, Stronger," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6517-6525, 2017.
26 Ren S, He K, Girshick R, et al., "Faster R-CNN: towards real-time object detection with region proposal networks," IEEE Trans Pattern Anal Mach Intell, vol. 39(6), pp.1137-1149, 2017.   DOI
27 Cai Z, Fan Q, Feris R S, et al., "A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection," in Proc. of European conference on computer vision, pp. 354-370, 2016.
28 Liu W, Anguelov D, Erhan D, et al., "SSD: Single Shot MultiBox Detector," in Proc. of European conference on computer vision, pp. 21-37, 2016.
29 Redmon J, Farhadi A., "YOLOv3: An Incremental Improvement," arXiv: Computer Vision and Pattern Recognition, 2018.
30 He K, Zhang X, Ren S, et al., "Deep Residual Learning for Image Recognition," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016.
31 Lecun Y, Boser B E, Denker J S, et al., "Backpropagation applied to handwritten zip code recognition," Neural Computation, vol. 1(4), pp. 541-551, 1989.   DOI
32 Zhang S, Wen L, Bian X, et al., "Single-Shot Refinement Neural Network for Object Detection," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4203-4212, 2018.
33 Ramana, L., Choi W., Cha, Y., "Fully automated vision-based loosened bolt detection using the Viola-Jones algorithm," Structural Health Monitoring-an International Journal, vol. 18, pp. 422-434, 2019.   DOI
34 Xia Y, Xie F, Jiang Z, et al., "Broken Railway Fastener Detection Based on Adaboost Algorithm," in Proc. of international conference on optoelectronics and image processing, pp. 313-316, 2010.
35 Mininath K. Nighot, Ashok Ghatol, Vilas M. Thakare, "Self-Organized Hybrid Wireless Sensor Network for Finding Randomly Moving Target in Unknown Environment," IJIMAI, vol. 5(1) , pp. 16-28, 2018.   DOI
36 Giben X, Patel V M, Chellappa R, et al., "Material classification and semantic segmentation of railway track images with deep convolutional neural networks," in Proc. of international conference on image processing, pp. 621-625, 2015.
37 Shelhamer, E.; Long, J. Darrell, T., "Fully Convolutional Networks for Semantic Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, pp. 640-651, 2017.   DOI
38 Dai J, Li Y, He K, et al., "R-FCN: Object Detection via Region-based Fully Convolutional Networks," in Proc. of the 30th International Conference on Neural Information Processing Systems, pp. 379-387, 2016.
39 Ravi S, Larochelle H., "Optimization as a Model for Few-Shot Learning," in Proc. of international conference on learning representations, 2017.
40 Gregory Koch Richard Zemel Salakhutdinov, R., "Siamese neural networks for one-shot image recognition," in Proc. of the 32nd International Conference on Machine Learning, 2015.
41 Shuai Liu, Xinyu Liu, Shuai Wang, et al., "Fuzzy-Aided Solution for Out-of-View Challenge in Visual Tracking under IoT Assisted Complex Environment," Neural Computing & Applications, vol. 33, no. 4, pp. 1055-1065, 2021.   DOI
42 Shuai Liu, Dongye Liu, Khan Muhammad, et al., "Effective Template Update Mechanism in Visual Tracking with Background Clutter," Neurocomputing, vol. 458, pp. 615-625, 2021.   DOI
43 Shuai Liu, Shuai Wang, Xinyu Liu, et al., "Human Memory Update Strategy: A Multi-Layer Template Update Mechanism for Remote Visual Monitoring," IEEE Transactions on Multimedia, 23, pp. 2188-2198, 2021.   DOI
44 Rezatofighi H, Tsoi N, Gwak J, et al., "Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 658-666, 2019.
45 Fefei L, Fergus, Perona, et al., "A Bayesian approach to unsupervised one-shot learning of object categories," in Proc. of international conference on computer vision, pp. 1134-1141, 2003.
46 Redmon J, Divvala S K, Girshick R, et al., "You Only Look Once: Unified, Real-Time Object Detection," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 779-788, 2016.