• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.10
• /
• pp.3729-3749
• /
• 2021

At present, deep convolution network-based salient object detection (SOD) has achieved impressive performance. However, it is still a challenging problem to make full use of the multi-scale information of the extracted features and which appropriate feature fusion method is adopted to process feature mapping. In this paper, we propose a new adjacency auxiliary network (AANet) based on multi-scale feature fusion for SOD. Firstly, we design the parallel connection feature enhancement module (PFEM) for each layer of feature extraction, which improves the feature density by connecting different dilated convolution branches in parallel, and add channel attention flow to fully extract the context information of features. Then the adjacent layer features with close degree of abstraction but different characteristic properties are fused through the adjacent auxiliary module (AAM) to eliminate the ambiguity and noise of the features. Besides, in order to refine the features effectively to get more accurate object boundaries, we design adjacency decoder (AAM_D) based on adjacency auxiliary module (AAM), which concatenates the features of adjacent layers, extracts their spatial attention, and then combines them with the output of AAM. The outputs of AAM_D features with semantic information and spatial detail obtained from each feature are used as salient prediction maps for multi-level feature joint supervising. Experiment results on six benchmark SOD datasets demonstrate that the proposed method outperforms similar previous methods.

• Journal of the Korea Institute of Military Science and Technology
• /
• v.12 no.4
• /
• pp.524-531
• /
• 2009

In this paper, we propose an automatic image registration method for multi-sensor image fusion such as visible and infrared images. The registration is achieved by finding corresponding feature points in both input images. In general, the global statistical correlation is not guaranteed between multi-sensor images, which bring out difficulties on the image registration for multi-sensor images. To cope with this problem, mutual information is adopted to measure correspondence of features and to select faithful points. An update algorithm for projective transform is also proposed. Experimental results show that the proposed method provides robust and accurate registration results.

• Advances in aircraft and spacecraft science
• /
• v.1 no.4
• /
• pp.455-469
• /
• 2014

High-precision autonomous localization technique is essential for future Mars rovers. This paper addresses an innovative integrated localization algorithm using a multiple information fusion approach. Firstly, the output of IMU is employed to construct the two-dimensional (2-D) dynamics equation of Mars rover. Secondly, radio beacon measurement and terrain image matching are considered as external measurements and included into the navigation filter to correct the inertial basis and drift. Then, extended Kalman filtering (EKF) algorithm is designed to estimate the position state of Mars rovers and suppress the measurement noise. Finally, the localization algorithm proposed in this paper is validated by computer simulation with different parameter sets.

• Current Optics and Photonics
• /
• v.8 no.3
• /
• pp.215-224
• /
• 2024

In medium- and long-range optical imaging systems, atmospheric turbulence causes blurring and distortion of images, resulting in loss of image information. An image-restoration method based on an adaptive feature-fusion multi-input-multi-output (MIMO) dense U-shaped network (Unet) is proposed, to restore a single image degraded by atmospheric turbulence. The network's model is based on the MIMO-Unet framework and incorporates patch-embedding shallow-convolution modules. These modules help in extracting shallow features of images and facilitate the processing of the multi-input dense encoding modules that follow. The combination of these modules improves the model's ability to analyze and extract features effectively. An asymmetric feature-fusion module is utilized to combine encoded features at varying scales, facilitating the feature reconstruction of the subsequent multi-output decoding modules for restoration of turbulence-degraded images. Based on experimental results, the adaptive feature-fusion MIMO dense U-shaped network outperforms traditional restoration methods, CMFNet network models, and standard MIMO-Unet network models, in terms of image-quality restoration. It effectively minimizes geometric deformation and blurring of images.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.10 no.6
• /
• pp.2801-2816
• /
• 2016

Owing to their low scalability, weak support on big data, insufficient data collaborative analysis and inadequate situational awareness, the traditional methods fail to meet the needs of the security data analysis. This paper proposes visualization methods to fuse the multi-source security data and grasp the network situation. Firstly, data sources are classified at their collection positions, with the objects of security data taken from three different layers. Secondly, the Heatmap is adopted to show host status; the Treemap is used to visualize Netflow logs; and the radial Node-link diagram is employed to express IPS logs. Finally, the Labeled Treemap is invented to make a fusion at data-level and the Time-series features are extracted to fuse data at feature-level. The comparative analyses with the prize-winning works prove this method enjoying substantial advantages for network analysts to facilitate data feature fusion, better understand network security situation with a unified, convenient and accurate mode.

• Journal of Satellite, Information and Communications
• /
• v.12 no.3
• /
• pp.15-21
• /
• 2017

Following the successful launch of KOMPSAT-3 in May 2012, KOMPSAT-5 in August 2013, and KOMPSAT-3A in March 2015 have succeeded in launching the integrated operation of optical, radar and thermal infrared sensors in Korea. We have established a foundation to utilize the characteristics of each sensors. In order to overcome limitations in the range of application and accuracy of the application of a single sensor, multi-sensor image fusion techniques have been developed which take advantage of multiple sensors and complement each other. In this paper, we introduce the development of software (InFusion) for multi-sensor image fusion and valued-added product generation using KOMPSAT series. First, we describe the characteristics of each sensor and the necessity of fusion software development, and describe the entire development process. It aims to increase the data utilization of KOMPSAT series and to inform the superiority of domestic software through creation of high value-added products.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.18 no.7
• /
• pp.1888-1906
• /
• 2024

Aiming at the problems that the edge of melanoma image is fuzzy, the contrast with the background is low, and the hair occlusion makes it difficult to segment accurately, this paper proposes a model MSCNet for melanoma segmentation based on U-net frame. Firstly, a multi-scale pyramid fusion module is designed to reconstruct the skip connection and transmit global information to the decoder. Secondly, the contextural information conduction module is innovatively added to the top of the encoder. The module provides different receptive fields for the segmented target by using the hole convolution with different expansion rates, so as to better fuse multi-scale contextural information. In addition, in order to suppress redundant information in the input image and pay more attention to melanoma feature information, global channel attention mechanism is introduced into the decoder. Finally, In order to solve the problem of lesion class imbalance, this paper uses a combined loss function. The algorithm of this paper is verified on ISIC 2017 and ISIC 2018 public datasets. The experimental results indicate that the proposed algorithm has better accuracy for melanoma segmentation compared with other CNN-based image segmentation algorithms.

• International Journal of Control, Automation, and Systems
• /
• v.4 no.2
• /
• pp.165-171
• /
• 2006

This paper proposes a data fusion algorithm of nonlinear multi sensor dynamic systems of synchronous sampling based on filtering step by step. Firstly, the object state variable at the next time index can be predicted by the previous global information with the systems, then the predicted estimation can be updated in turn by use of the extended Kalman filter when all of the observations aiming at the target state variable arrive. Finally a fusion estimation of the object state variable is obtained based on the system global information. Synchronously, we formulate the new algorithm and compare its performances with those of the traditional nonlinear centralized and distributed data fusion algorithms by the indexes that include the computational complexity, data communicational burden, time delay and estimation accuracy, etc.. These compared results indicate that the performance from the new algorithm is superior to the performances from the two traditional nonlinear data fusion algorithms.

• Smart Structures and Systems
• /
• v.23 no.3
• /
• pp.279-293
• /
• 2019

To ensure high quality data being used for data mining or feature extraction in the bridge structural health monitoring (SHM) system, a practical sensor fault diagnosis methodology has been developed based on the similarity of symmetric structure responses. First, the similarity of symmetric response is discussed using field monitoring data from different sensor types. All the sensors are initially paired and sensor faults are then detected pair by pair to achieve the multi-fault diagnosis of sensor systems. To resolve the coupling response issue between structural damage and sensor fault, the similarity for the target zone (where the studied sensor pair is located) is assessed to determine whether the localized structural damage or sensor fault results in the dissimilarity of the studied sensor pair. If the suspected sensor pair is detected with at least one sensor being faulty, field test could be implemented to support the regression analysis based on the monitoring and field test data for sensor fault isolation and reconstruction. Finally, a case study is adopted to demonstrate the effectiveness of the proposed methodology. As a result, Dasarathy's information fusion model is adopted for multi-sensor information fusion. Euclidean distance is selected as the index to assess the similarity. In conclusion, the proposed method is practical for actual engineering which ensures the reliability of further analysis based on monitoring data.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.17 no.1
• /
• pp.51-73
• /
• 2023

Social media play a significant role in communicating information across the globe, connecting with loved ones, getting the news, communicating ideas, etc. However, a group of people uses social media to spread fake information, which has a bad impact on society. Therefore, minimizing fake news and its detection are the two primary challenges that need to be addressed. This paper presents a multi-modal deep learning technique to address the above challenges. The proposed modal can use and process visual and textual features. Therefore, it has the ability to detect fake information from visual and textual data. We used EfficientNetB0 and a sentence transformer, respectively, for detecting counterfeit images and for textural learning. Feature embedding is performed at individual channels, whilst fusion is done at the last classification layer. The late fusion is applied intentionally to mitigate the noisy data that are generated by multi-modalities. Extensive experiments are conducted, and performance is evaluated against state-of-the-art methods. Three real-world benchmark datasets, such as MediaEval (Twitter), Weibo, and Fakeddit, are used for experimentation. Result reveals that the proposed modal outperformed the state-of-the-art methods and achieved an accuracy of 86.48%, 82.50%, and 88.80%, respectively, for MediaEval (Twitter), Weibo, and Fakeddit datasets.