Browse > Article
http://dx.doi.org/10.3837/tiis.2020.08.024

Weather Recognition Based on 3C-CNN  

Tan, Ling (School of Computer and Software, Nanjing University of Information Science and Technology)
Xuan, Dawei (School of Artificial Intelligence, Nanjing University of Information Science and Technology)
Xia, Jingming (School of Artificial Intelligence, Nanjing University of Information Science and Technology)
Wang, Chao (Lab of Environmental Modeling and Spatial Analysis, The Ohio State University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.14, no.8, 2020 , pp. 3567-3582 More about this Journal
Abstract
Human activities are often affected by weather conditions. Automatic weather recognition is meaningful to traffic alerting, driving assistance, and intelligent traffic. With the boost of deep learning and AI, deep convolutional neural networks (CNN) are utilized to identify weather situations. In this paper, a three-channel convolutional neural network (3C-CNN) model is proposed on the basis of ResNet50.The model extracts global weather features from the whole image through the ResNet50 branch, and extracts the sky and ground features from the top and bottom regions by two CNN5 branches. Then the global features and the local features are merged by the Concat function. Finally, the weather image is classified by Softmax classifier and the identification result is output. In addition, a medium-scale dataset containing 6,185 outdoor weather images named WeatherDataset-6 is established. 3C-CNN is used to train and test both on the Two-class Weather Images and WeatherDataset-6. The experimental results show that 3C-CNN achieves best on both datasets, with the average recognition accuracy up to 94.35% and 95.81% respectively, which is superior to other classic convolutional neural networks such as AlexNet, VGG16, and ResNet50. It is prospected that our method can also work well for images taken at night with further improvement.
Keywords
Weather recognition; deep learning; ResNet50; 3C-CNN; WeatherDataset-6;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Zhang, Mengmeng, Wei Li, and Qian Du. "Diverse region-based CNN for hyperspectral image classification," IEEE Transactions on Image Processing, 27(6), 2623-2634, 2018.   DOI
2 Zhao, Bin, et al., "A CNN-RNN architecture for multi-label weather recognition," Neurocomputing, 322, 47-57, 2018.   DOI
3 Loven, Lauri, et al., "Mobile road weather sensor calibration by sensor fusion and linear mixed models," PloS one, 14(2), 2019.
4 Chu, Wei-Ta, Xiang-You Zheng, and Ding-Shiuan Ding, "Camera as weather sensor: Estimating weather information from single images," Journal of Visual Communication and Image Representation, 46, 233-249, 2017.   DOI
5 Ponce, Juan Manuel, et al., "Automatic counting and individual size and mass estimation of olive-fruits through computer vision techniques," IEEE Access, 7, 59451-59465, 2019.   DOI
6 Voulodimos, Athanasios, et al. "Deep learning for computer vision: A brief review," Computational intelligence and neuroscience, Vol. 2018, 2018.
7 Lin, Di, et al. "RSCM: Region selection and concurrency model for multi-class weather recognition," IEEE Transactions on Image Processing, 26(9), 4154-4167, 2017.   DOI
8 Wu, Xianyu, et al. "Text Detection and Recognition for Natural Scene Images Using Deep Convolutional Neural Networks," Computers, Materials & Continua, Vol. 61, No. 1, 289-300, 2019.   DOI
9 Wu, Jianxin, and Jim M. Rehg. "Centrist: A visual descriptor for scene categorization," IEEE transactions on pattern analysis and machine intelligence, 33(8), 1489-1501, 2010.   DOI
10 Van De Sande, Koen, Theo Gevers, and Cees Snoek. "Evaluating color descriptors for object and scene recognition," IEEE transactions on pattern analysis and machine intelligence, 32(9), 1582-1596, 2009.   DOI
11 Guerrero-Ibanez, Juan, Sherali Zeadally, and Juan Contreras-Castillo. "Sensor technologies for intelligent transportation systems," Sensors, 18(4), 1212, 2018.   DOI
12 Lin, Di, et al., "Learning important spatial pooling regions for scene classification," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
13 Lagorio, Andrea, Enrico Grosso, and Massimo Tistarelli, "Automatic detection of adverse weather conditions in traffic scenes," in Proc. of 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance, IEEE, 2008.
14 Shi, Yuzhou, et al., "Weather Recognition Based on Edge Deterioration and Convolutional Neural Networks," in Proc. of 2018 24th International Conference on Pattern Recognition (ICPR), IEEE, 2018.
15 Chen, Ken, et al., "An Early Warning Method for Highway Traffic Accidents Based on Bayesian Networks," in Proc. of the 2019 International Conference on Artificial Intelligence and Computer Science, 2019.
16 Hsieh, Ping-Jui, et al., "Path Planning Design for Vehicle Driving Assist Systems," in Proc. of 2018 International Conference on System Science and Engineering (ICSSE), IEEE, 2018.
17 Dannheim, Clemens, et al., "Weather detection in vehicles by means of camera and lidar systems," in Proc. of 2014 Sixth International Conference on Computational Intelligence, Communication Systems and Networks, IEEE, 2014.
18 Chen, Zichong, et al., "Howis the weather: Automatic inference from images," in Proc. of 2012 19th IEEE International Conference on Image Processing, IEEE, 2012.
19 Chen, Yuantao, et al., "The fire recognition algorithm using dynamic feature fusion and IV-SVM classifier," Cluster Computing, 22(3), 7665-7675, 2019.   DOI
20 Li, Qian, Yi Kong, and Shi-ming Xia, "A method of weather recognition based on outdoor images," in Proc. of 2014 International Conference on Computer Vision Theory and Applications (VISAPP). IEEE, Vol. 2, 2014.
21 Li, Ruixiang, et al., "Street-Level Landmarks Acquisition Based on SVM Classifiers," CMC-Computers Materlals & Continua, 59(2), 591-606, 2019.   DOI
22 Narasimhan, Srinivasa G., Chi Wang, and Shree K. Nayar, "All the images of an outdoor scene," in Proc. of European conference on computer vision, Springer, Berlin, Heidelberg, 2002.
23 Lu, Cewu, et al., "Two-class weather classification," in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2014.
24 Zhang, Zheng, and Huadong Ma, "Multi-class weather classification on single images," in Proc. of 2015 IEEE International Conference on Image Processing (ICIP), IEEE, 2015.
25 Zhang, Zheng, et al., "Scene-free multi-class weather classification on single images," Neurocomputing, 207, 365-37, 2016.   DOI
26 Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems, 2012.
27 Simonyan, Karen, and Andrew Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv, 1409-1556, 2014.
28 Tang, Pengjie, Hanli Wang, and Sam Kwong, "G-MS2F: GoogLeNet based multi-stage feature fusion of deep CNN for scene recognition," Neurocomputing, 225, 188-197, 2017.   DOI
29 He, Kaiming, et al., "Deep residual learning for image recognition," in Proc. of the IEEE conference on computer vision and pattern recognition, 2016.
30 Zeng, Daojian, et al., "Aspect based sentiment analysis by a linguistically regularized CNN with gated mechanism," Journal of Intelligent & Fuzzy Systems, 36(5), 3971-3980, 2019.   DOI
31 Liu, Jin, et al., "Attention-based BiGRU-CNN for Chinese question classification," Journal of Ambient Intelligence and Humanized Computing, 1-12, 2019.
32 Zhang, Jianming, et al., "Spatial and semantic convolutional features for robust visual object tracking," Multimedia Tools and Applications, 1-21, 2018.
33 Su, Jian, et al., "A time and energy saving based frame adjustment strategy (TES-FAS) tag identification algorithm for UHF RFID systems," IEEE Transactions on Wireless Communications, 2020.
34 Pan, Lili, et al., "Image augmentation-based food recognition with convolutional neural networks," Comput. Mater. Continua, 59(1), 297-313, 2019.   DOI
35 Xia, Zhihua, et al., "A privacy-preserving image retrieval based on AC-coefficients and color histograms in cloud environment," Computers, Materials & Continua, 58(1), 27-44, 2019.   DOI
36 Tan, Ling, and Xin Yu, "Medical image fusion based on fast finite shearlet transform and sparse representation," Computational and mathematical methods in medicine, Vol. 2019, 2019.
37 Chen, Liang-Chieh, et al., "Encoder-decoder with atrous separable convolution for semantic image segmentation," in Proc. of the European conference on computer vision (ECCV), 2018.
38 Zhu, Ziqi, et al. "Extreme weather recognition using a novel fine-tuning strategy and optimized GoogLeNet," in Proc. of 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), IEEE, 2017.
39 Elhoseiny, Mohamed, Sheng Huang, and Ahmed Elgammal, "Weather classification with deep convolutional neural networks," in Proc. of 2015 IEEE International Conference on Image Processing (ICIP), IEEE, 2015.
40 Lu, Cewu, et al., "Two-class weather classification," IEEE Transactions on Pattern Analysis and Machine Intelligence, 39 (12), 2510-2524, 2017.   DOI
41 Zhu, Ziqi, et al., "Extreme weather recognition using convolutional neural networks," in Proc. of 2016 IEEE International Symposium on Multimedia (ISM), IEEE, 2016.
42 Xiao, Jianxiong, et al., "Sun database: Large-scale scene recognition from abbey to zoo," in Proc. of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, 2010.
43 Russell, Bryan C., et al., "LabelMe: a database and web-based tool for image annotation," International journal of computer vision, 77.1-3, 157-173, 2008.   DOI
44 Salamon, Justin, and Juan Pablo Bello, "Deep convolutional neural networks and data augmentation for environmental sound classification," IEEE Signal Processing Letters, 24(3), 279-283, 2017.   DOI
45 Russakovsky, Olga, et al., "Imagenet large scale visual recognition challenge," International journal of computer vision, 115(3), 211-252, 2015.   DOI