• KIPS Transactions on Software and Data Engineering
• /
• v.12 no.5
• /
• pp.217-228
• /
• 2023

Video instance segmentation is an intelligent visual task with high complexity because it not only requires object instance segmentation for each image frame constituting a video, but also requires accurate tracking of instances throughout the frame sequence of the video. In special, human instance segmentation in drama videos has an unique characteristic that requires accurate tracking of several main characters interacting in various places and times. Also, it is also characterized by a kind of the class imbalance problem because there is a significant difference between the frequency of main characters and that of supporting or auxiliary characters in drama videos. In this paper, we introduce a new human instance datatset called MHIS, which is built upon drama videos, Miseang, and then propose a novel video data augmentation method, CDVA, in order to overcome the data imbalance problem between character classes. Different from the previous video data augmentation methods, the proposed CDVA generates more realistic augmented videos by deciding the optimal location within the background clip for a target human instance to be inserted with taking rich spatio-temporal context embedded in videos into account. Therefore, the proposed augmentation method, CDVA, can improve the performance of a deep neural network model for video instance segmentation. Conducting both quantitative and qualitative experiments using the MHIS dataset, we prove the usefulness and effectiveness of the proposed video data augmentation method.

• Journal of The Korean Society of Agricultural Engineers
• /
• v.59 no.1
• /
• pp.57-70
• /
• 2017

In this study, we evaluated daily root zone soil moisture dynamics and agricultural drought using a near-surface soil moisture data assimilation scheme with Soil Moisture Active & Passive (SMAP, $3km{\times}3km$) soil moisture footprints under different hydro-climate conditions. Satellite-based LANDSAT and MODIS image footprints were converted to spatially-distributed soil moisture estimates based on the regression model, and the converted soil moisture distributions were used for assessing uncertainties and applicability of SMAP data at fields. In order to overcome drawbacks of the discontinuity of SMAP data at the spatio-temporal scales, the data assimilation was applied to SMAP for estimating daily soil moisture dynamics at the spatial domain. Then, daily soil moisture values were used to estimate weekly agricultural drought based on the Soil Moisture Deficit Index (SMDI). The Yongdam-dam and Soyan river-dam watersheds were selected for validating our proposed approach. As a results, the MODIS/SMAP soil moisture values were relatively overestimated compared to those of the TDR-based measurements and LANDSAT data. When we applied the data assimilation scheme to SMAP, uncertainties were highly reduced compared to the TDR measurements. The estimated daily root zone soil moisture dynamics and agricultural drought from SMAP showed the variability at the sptio-temporal scales indicating that soil moisture values are influenced by not only the precipitation, but also the land surface characteristics. These findings can be useful for establishing efficient water management plans in hydrology and agricultural drought.

• Journal of KIISE:Software and Applications
• /
• v.31 no.9
• /
• pp.1142-1149
• /
• 2004

Motion Estimation(ME) has been developed to reduce temporal redundancy in digital video signals and increase data compression ratio. ME is an Important part of video encoding systems, since it can significantly affect the output quality of encoded sequences. However, ME requires high computational complexity, it is difficult to apply to real time video transmission. for this reason, motion estimation algorithms with low computational complexity are viable solutions. In this paper, we present an efficient method with low computational complexity based on spatial and temporal correlations of motion vectors. The proposed method uses temporally and spatially correlated motion information, the motion vector of the block with the same coordinate in the reference frame and the motion vectors of neighboring blocks around the current block in the current frame, to decide the search pattern and the location of search starting point adaptively. Experiments show that the image quality improvement of the proposed method over MVFAST (Motion Vector Field Adaptive Search Technique) and PMVFAST (Predictive Motion Vector Field Adaptive Search Technique) is 0.01～0.3(dB) better and the speedup improvement is about 1.12～l.33 times faster which resulted from lower computational complexity.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.43 no.2 s.308
• /
• pp.1-11
• /
• 2006

In this paper, we propose the novel video retrieval algorithm using non-parametric based motion classification in the shot-based video indexing structure. The proposed system firstly gets the key frame and motion information from each shot segmented by scene change detection method, and then extracts visual features and non-parametric based motion information from them. Finally, we construct real-time retrieval system supporting similarity comparison of these spatio-temporal features. After the normalized motion vector fields is created from MPEG compressed stream, the extraction of non-parametric based motion feature is effectively achieved by discretizing each normalized motion vectors into various angle bins, and considering a mean, a variance, and a direction of these bins. We use the edge-based spatial descriptor to extract the visual feature in key frames. Experimental evidence shows that our algorithm outperforms other video retrieval methods for image indexing and retrieval. To index the feature vectors, we use R*-tree structures.

• Cartoon and Animation Studies
• /
• s.49
• /
• pp.203-220
• /
• 2017

In the field of visual arts, the introduction of synthesis allowed us to express various expressions that were outside the constraints of time and space. Digital synthesis is constrained by realistic representation due to technical limitations. However, it is no exaggeration to say that after digital synthesis has been fully established, at least the limits of image representation through synthesis have almost disappeared. The existing research papers on composing are either technical studies on the production techniques, or the synthesis was mainly focused on the change of the space-time concept and meaning of the visual arts. I felt the need for research on. In addition, I felt the need to look back on the changes in my view of art. Therefore, in this paper, rather than dealing with the conceptual theory of the technical part or spatio-temporal extension, it is necessary to classify it as natural, heterogeneous, heterogeneous and natural according to the traits revealed in the artistic expression of art, And diversity.

• Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
• /
• v.40 no.4
• /
• pp.275-292
• /
• 2022

In order to detect climate changes using satellite imagery, the GCOS (Global Climate Observing System) defines requirements such as spatio-temporal resolution, stability by the time change, and uncertainty. Due to limitation of GK-2A sensor performance, the level-2 products can not satisfy the requirement, especially for spatial resolution. In this paper, we found the optimal pan-sharpening algorithm for GK-2A products. The six pan-sharpening methods included in CS (Component Substitution), MRA (Multi-Resolution Analysis), VO (Variational Optimization), and DL (Deep Learning) were used. In the case of DL, the synthesis property based method was used to generate training dataset. The process of synthesis property is that pan-sharpening model is applied with Pan (Panchromatic) and MS (Multispectral) images with reduced spatial resolution, and fused image is compared with the original MS image. In the synthesis property based method, fused image with desire level for user can be produced only when the geometric characteristics between the PAN with reduced spatial resolution and MS image are similar. However, since the dissimilarity exists, RD (Random Down-sampling) was additionally used as a way to minimize it. Among the pan-sharpening methods, PSGAN was applied with RD (PSGAN_RD). The fused images are qualitatively and quantitatively validated with consistency property and the synthesis property. As validation result, the GSA algorithm performs well in the evaluation index representing spatial characteristics. In the case of spectral characteristics, the PSGAN_RD has the best accuracy with the original MS image. Therefore, in consideration of spatial and spectral characteristics of fused image, we found that PSGAN_RD is suitable for GK-2A products.

• Korean Journal of Remote Sensing
• /
• v.37 no.6_1
• /
• pp.1507-1515
• /
• 2021

If chlorophyll-a is estimated through ocean color remote sensing, it is able to understand the global distribution of phytoplankton and primary production. However, there are missing data in the ocean color observed from the satellites due to the clouds or weather conditions. In thisstudy, the missing data of the GOCI (Geostationary Ocean Color Imager) chlorophyll-a product wasreconstructed by using DINEOF (Data INterpolation Empirical Orthogonal Functions). DINEOF reconstructs the missing data based on spatio-temporal data, and the accuracy was cross-verified by removing a part of the GOCI chlorophyll-a image and comparing it with the reconstructed image. In the study area, the optimal EOF (Empirical Orthogonal Functions) mode for DINEOF wasin 10-13. The temporal and spatialreconstructed data reflected the increasing chlorophyll-a concentration in the afternoon, and the noise of outliers was filtered. Therefore, it is expected that DINEOF is useful to reconstruct the missing images, also it is considered that it is able to use as basic data for monitoring the ocean environment.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.23 no.3
• /
• pp.145-151
• /
• 2023

3D-CNN is one of the deep learning techniques for learning time series data. Such three-dimensional learning can generate many parameters, so that high-performance machine learning is required or can have a large impact on the learning rate. When learning dynamic hand-gestures in spatiotemporal domain, it is necessary for the improvement of the efficiency of dynamic hand-gesture learning with 3D-CNN to find the optimal conditions of input video data by analyzing the learning accuracy according to the spatiotemporal change of input video data without structural change of the 3D-CNN model. First, the time ratio between dynamic hand-gesture actions is adjusted by setting the learning interval of image frames in the dynamic hand-gesture video data. Second, through 2D cross-correlation analysis between classes, similarity between image frames of input video data is measured and normalized to obtain an average value between frames and analyze learning accuracy. Based on this analysis, this work proposed two methods to effectively select input video data for 3D-CNN deep learning of dynamic hand-gestures. Experimental results showed that the learning interval of image data frames and the similarity of image frames between classes can affect the accuracy of the learning model.

• The Journal of the Korea Contents Association
• /
• v.7 no.2
• /
• pp.40-50
• /
• 2007

In this paper, we propose an efficient method for detecting and tracking multiple moving objects based on morphological region merging from real-time video sequences. The proposed approach consists of adaptive threshold extraction, morphological region merging and detecting and tracking of objects. Firstly, input frame is separated into moving regions and static regions using the difference of images between two consecutive frames. Secondly, objects are segmented with a reference background image and adaptive threshold values, then, the segmentation result is refined by morphological region merge algorithm. Lastly, each object segmented in a previous step is assigned a consistent identification over time, based on its spatio-temporal information. The experimental results show that a proposed method is efficient and useful in terms of real-time multiple objects detecting and tracking.

• The Journal of the Korea Contents Association
• /
• v.12 no.12
• /
• pp.597-604
• /
• 2012

In order to actualize the third dimension form, hologram is coming to attention because it has no restriction on viewing position and is capable of natural visual expression. Although hologram technology is the best method to embody 3D image without glasses, it is not commercialized due to several technological problems. Currently used hologram technology in concerts or exhibitions are images flashed on a 2-dimensional transparent screen by HD projectors which is similar to hologram technology, not truly same. In this research, we make 3D contents for Holographic projection and use these contents to present art that can interact with spectators. As a result of the exhibition, attendance showed satisfaction on inspection form, allowing spectators to move around the screen and view it both sides; moreover, they were enterprising to interact with the videos played according to their movements. Therefore, we are able to implement a sensible and spatio-temporal artwork along with interesting space production and represent a intimate and interactive space with the public.