• Korean Journal of Remote Sensing
• /
• v.36 no.5_1
• /
• pp.807-821
• /
• 2020

The objective of this study is to evaluate the applicability of representative spatio-temporal fusion models developed for the fusion of mid- and low-resolution satellite images in order to construct a set of time-series high-resolution images for crop monitoring. Particularly, the effects of the characteristics of input image pairs on the prediction performance are investigated by considering the principle of spatio-temporal fusion. An experiment on the fusion of multi-temporal Sentinel-2 and RapidEye images in agricultural fields was conducted to evaluate the prediction performance. Three representative fusion models, including Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM), SParse-representation-based SpatioTemporal reflectance Fusion Model (SPSTFM), and Flexible Spatiotemporal DAta Fusion (FSDAF), were applied to this comparative experiment. The three spatio-temporal fusion models exhibited different prediction performance in terms of prediction errors and spatial similarity. However, regardless of the model types, the correlation between coarse resolution images acquired on the pair dates and the prediction date was more significant than the difference between the pair dates and the prediction date to improve the prediction performance. In addition, using vegetation index as input for spatio-temporal fusion showed better prediction performance by alleviating error propagation problems, compared with using fused reflectance values in the calculation of vegetation index. These experimental results can be used as basic information for both the selection of optimal image pairs and input types, and the development of an advanced model in spatio-temporal fusion for crop monitoring.

• International journal of advanced smart convergence
• /
• v.12 no.4
• /
• pp.88-97
• /
• 2023

Traffic flow prediction is of great significance in urban planning and traffic management. As the complexity of urban traffic increases, existing prediction methods still face challenges, especially for the fusion of spatiotemporal information and the capture of long-term dependencies. This study aims to use the fusion model of graph neural network to solve the spatio-temporal information fusion problem in traffic flow prediction. We propose a new deep learning model Spatio-Temporal Information Fusion using Graph Neural Networks (STFGNN). We use GCN module, TCN module and LSTM module alternately to carry out spatiotemporal information fusion. GCN and multi-core TCN capture the temporal and spatial dependencies of traffic flow respectively, and LSTM connects multiple fusion modules to carry out spatiotemporal information fusion. In the experimental evaluation of real traffic flow data, STFGNN showed better performance than other models.

• Journal of the Korean Society of Food Culture
• /
• v.23 no.2
• /
• pp.204-213
• /
• 2008

In this age of information ruled by new technologies and knowledge, the world is interconnected as a single community, and within this trend of globalization, new cultural codes are emerging through temporal fusion between the past and the present and spatial fusion between different regions and countries. In this situation, it seems meaningful to review Korean fusion foods and restaurants serving such foods, as well as to consider their future directions. Thus, the objective of the present study was to survey and analyze Korean fusion restaurants representing Korean culture not only in Korea, but also in foreign cities, and to identify the expression characteristics of such restaurants. Based on restaurants recommended in relevant magazines and on Internet sites, 18 spaces were selected, visited, and surveyed, in which tradition and modernism were well-mixed. Data on the shapes, materials, colors, and patterns of spatial elements and food-related elements, including photographs, were collected and analyzed. The results are as follows. Of the 18 restaurants, 13 (72%) showed temporal fusion in both spatial and food-related elements, 4 showed temporal fusion in spatial elements and cultural fusion in food-related elements, and 1 showed cultural fusion in both spatial and food-related elements. In general, the spaces were mainly designed with modern elements and partially with traditional elements (ceilings, windows, furniture, articles), and the fusion of food-related elements was made in diverse forms that included temporal fusion restructuring traditional menus contemporarily, and cultural fusion harmonizing traditional food with Western cookery.

• Korean Journal of Remote Sensing
• /
• v.35 no.6_3
• /
• pp.1209-1219
• /
• 2019

For consistent vegetation monitoring, it is necessary to generate time-series vegetation index datasets at fine temporal and spatial scales by fusing the complementary characteristics between temporal and spatial scales of multiple satellite data. In this study, we quantitatively and qualitatively analyzed the prediction accuracy of time-series change information extracted from spatio-temporal fusion models of multiple satellite data for vegetation monitoring. As for the spatio-temporal fusion models, we applied two models that have been widely employed to vegetation monitoring, including a Spatial and Temporal Adaptive Reflectance Fusion Model (STARFM) and an Enhanced Spatial and Temporal Adaptive Reflectance Fusion Model (ESTARFM). To quantitatively evaluate the prediction accuracy, we first generated simulated data sets from MODIS data with fine temporal scales and then used them as inputs for the spatio-temporal fusion models. We observed from the comparative experiment that ESTARFM showed better prediction performance than STARFM, but the prediction performance for the two models became degraded as the difference between the prediction date and the simultaneous acquisition date of the input data increased. This result indicates that multiple data acquired close to the prediction date should be used to improve the prediction accuracy. When considering the limited availability of optical images, it is necessary to develop an advanced spatio-temporal model that can reflect the suggestions of this study for vegetation monitoring.

• Korean Journal of Remote Sensing
• /
• v.37 no.6_3
• /
• pp.1931-1942
• /
• 2021

As the utility of an optical satellite image with a high spatial resolution (i.e., fine-scale) has been emphasized, recently, various studies of the land surface monitoring using those have been widely carried out. However, the usefulness of fine-scale satellite images is limited because those are acquired at a low temporal resolution. To compensate for this limitation, the spatiotemporal data fusion can be applied to generate a synthetic image with a high spatio-temporal resolution by fusing multiple satellite images with different spatial and temporal resolutions. Since the spatio-temporal data fusion models have been developed for mid or low spatial resolution satellite images in the previous studies, it is necessary to evaluate the applicability of the developed models to the satellite images with a high spatial resolution. For this, this study evaluated the applicability of the developed spatio-temporal fusion models for KOMPSAT-3A and Sentinel-2 images. Here, an Enhanced Spatial and Temporal Adaptive Fusion Model (ESTARFM) and Spatial Time-series Geostatistical Deconvolution/Fusion Model (STGDFM), which use the different information for prediction, were applied. As a result of this study, it was found that the prediction performance of STGDFM, which combines temporally continuous reflectance values, was better than that of ESTARFM. Particularly, the prediction performance of STGDFM was significantly improved when it is difficult to simultaneously acquire KOMPSAT and Sentinel-2 images at a same date due to the low temporal resolution of KOMPSAT images. From the results of this study, it was confirmed that STGDFM, which has relatively better prediction performance by combining continuous temporal information, can compensate for the limitation to the low revisit time of fine-scale satellite images.

• Proceedings of the KSRS Conference
• /
• 2005.10a
• /
• pp.434-436
• /
• 2005

Analysis method using remote sensing data is one of the effective ways to research a spatial and temporal variability of the mesoscale oceanic motions. During past several decades, many researchers have been getting comprehensive results using remote sensing data with application of data fusion methods in many parts of geo-science. For this study, we took the integration and fusion of several remote sensing data, which are different data resolution, timescale and characteristics, for improving accurate analysis of variation of the Kuroshio Extension. Furthermore, we might get advanced ways to understand the variability of the Kuroshio Extension, has close relation to the spatial and temporal variation of the Kuroshio and Oyashio Current.

• Korean Journal of Remote Sensing
• /
• v.21 no.2
• /
• pp.145-162
• /
• 2005

To improve the accuracy of land-cover discrimination in SAB data classification, this paper presents a methodology that includes feature extraction and fusion steps with multi-temporal SAR data. Three features including average backscattering coefficient, temporal variability and coherence are extracted from multi-temporal SAR data by considering the temporal behaviors of backscattering characteristics of SAR sensors. Dempster-Shafer theory of evidence(D-S theory) and fuzzy logic are applied to effectively integrate those features. Especially, a feature-driven heuristic approach to mass function assignment in D-S theory is applied and various fuzzy combination operators are tested in fuzzy logic fusion. As experimental results on a multi-temporal Radarsat-1 data set, the features considered in this paper could provide complementary information and thus effectively discriminated water, paddy and urban areas. However, it was difficult to discriminate forest and dry fields. From an information fusion methodological point of view, the D-S theory and fuzzy combination operators except the fuzzy Max and Algebraic Sum operators showed similar land-cover accuracy statistics.

• Korean Journal of Remote Sensing
• /
• v.37 no.5_1
• /
• pp.999-1011
• /
• 2021

In spatio-temporal fusion aiming at predicting images with both high spatial and temporal resolutionsfrom multi-sensor images, the radiometric inconsistency between input multi-sensor images may affect prediction performance. This study investigates the effect of radiometric correction, which compensate different spectral responses of multi-sensor satellite images, on the spatio-temporal fusion results. The effect of relative radiometric correction of input images was quantitatively analyzed through the case studies using Sentinel-2, PlanetScope, and RapidEye images obtained from two croplands. Prediction performance was improved when radiometrically corrected multi-sensor images were used asinput. In particular, the improvement in prediction performance wassubstantial when the correlation between input images was relatively low. Prediction performance could be improved by transforming multi-sensor images with different spectral responses into images with similar spectral responses and high correlation. These results indicate that radiometric correction is required to improve prediction performance in spatio-temporal fusion of multi-sensor satellite images with low correlation.

• KIPS Transactions on Software and Data Engineering
• /
• v.11 no.2
• /
• pp.81-86
• /
• 2022

Given that time series are used in various fields, such as finance, IoT, and manufacturing, data analytical methods for accurate time-series forecasting can serve to increase operational efficiency. Among time-series analysis methods, multi-horizon forecasting provides a better understanding of data because it can extract meaningful statistics and other characteristics of the entire time-series. Furthermore, time-series data with exogenous information can be accurately predicted by using multi-horizon forecasting methods. However, traditional deep learning-based models for time-series do not account for the heterogeneity of inputs. We proposed an improved time-series predicting method, called the temporal fusion transformer method, which combines multi-horizon forecasting with interpretable insights into temporal dynamics. Various real-world data such as stock prices, fine dust concentrates and electricity consumption were considered in experiments. Experimental results showed that our temporal fusion transformer method has better time-series forecasting performance than existing models.

• The KIPS Transactions:PartB
• /
• v.18B no.6
• /
• pp.365-374
• /
• 2011

Recently, malicious video classification and filtering techniques are of practical interest as ones can easily access to malicious multimedia contents through the Internet, IPTV, online social network, and etc. Considerable research efforts have been made to developing malicious video classification and filtering systems. However, the malicious video classification and filtering is not still being from mature in terms of reliable classification/filtering performance. In particular, the most of conventional approaches have been limited to using only the spatial features (such as a ratio of skin regions and bag of visual words) for the purpose of malicious image classification. Hence, previous approaches have been restricted to achieving acceptable classification and filtering performance. In order to overcome the aforementioned limitation, we propose new malicious video classification framework that takes advantage of using both the spatial and temporal features that are readily extracted from a sequence of video frames. In particular, we develop the effective temporal features based on the motion periodicity feature and temporal correlation. In addition, to exploit the best data fusion approach aiming to combine the spatial and temporal features, the representative data fusion approaches are applied to the proposed framework. To demonstrate the effectiveness of our method, we collect 200 sexual intercourse videos and 200 non-sexual intercourse videos. Experimental results show that the proposed method increases 3.75% (from 92.25% to 96%) for classification of sexual intercourse video in terms of accuracy. Further, based on our experimental results, feature-level fusion approach (for fusing spatial and temporal features) is found to achieve the best classification accuracy.