• ETRI Journal
• /
• v.44 no.2
• /
• pp.327-338
• /
• 2022

Classification models for human action recognition require robust features and large training sets for good generalization. However, data augmentation methods are employed for imbalanced training sets to achieve higher accuracy. These samples generated using data augmentation only reflect existing samples within the training set, their feature representations are less diverse and hence, contribute to less precise classification. This paper presents new data augmentation and action representation approaches to grow training sets. The proposed approach is based on two fundamental concepts: virtual video generation for augmentation and representation of the action videos through robust features. Virtual videos are generated from the motion history templates of action videos, which are convolved using a convolutional neural network, to generate deep features. Furthermore, by observing an objective function of the genetic algorithm, the spatiotemporal features of different samples are combined, to generate the representations of the virtual videos and then classified through an extreme learning machine classifier on MuHAVi-Uncut, iXMAS, and IAVID-1 datasets.

• KIPS Transactions on Software and Data Engineering
• /
• v.12 no.5
• /
• pp.217-228
• /
• 2023

Video instance segmentation is an intelligent visual task with high complexity because it not only requires object instance segmentation for each image frame constituting a video, but also requires accurate tracking of instances throughout the frame sequence of the video. In special, human instance segmentation in drama videos has an unique characteristic that requires accurate tracking of several main characters interacting in various places and times. Also, it is also characterized by a kind of the class imbalance problem because there is a significant difference between the frequency of main characters and that of supporting or auxiliary characters in drama videos. In this paper, we introduce a new human instance datatset called MHIS, which is built upon drama videos, Miseang, and then propose a novel video data augmentation method, CDVA, in order to overcome the data imbalance problem between character classes. Different from the previous video data augmentation methods, the proposed CDVA generates more realistic augmented videos by deciding the optimal location within the background clip for a target human instance to be inserted with taking rich spatio-temporal context embedded in videos into account. Therefore, the proposed augmentation method, CDVA, can improve the performance of a deep neural network model for video instance segmentation. Conducting both quantitative and qualitative experiments using the MHIS dataset, we prove the usefulness and effectiveness of the proposed video data augmentation method.

• Journal of Broadcast Engineering
• /
• v.27 no.1
• /
• pp.13-19
• /
• 2022

Video Object Segmentation (VOS) task requires an amount of labeled sequence data, which limits the performance of the current VOS methods trained with public datasets. In this paper, we propose two effective data augmentation schemes for VOS. The first augmentation method is to swap the background segment to the background from another image, and the other method is to play the sequence in reverse. The two augmentation schemes for VOS enable the current VOS methods to robustly predict the segmentation labels and improve the performance of VOS.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 1998.06b
• /
• pp.147-153
• /
• 1998

This paper provides a method for video augmentation using image interpolation. In computer graphics or augmented reality, 3D information of a model object is necessary to generate 2D views of the model, which are then inserted into or overlayed on environmental views or real video frames. However, we do not require any three dimensional model but images of the model object at some locations to render views according to the motion of video camera which is calculated by an SFM algorithm using point matches under weak-perspective (scaled-orthographic) projection model. Thus, a linear view interpolation algorithm is applied rather than a 3D ray-tracing method to get a view of the model at different viewpoints from model views. In order to get novel views in a way that agrees with the camera motion the camera coordinate system is embedded into model coordinate system at initialization time on the basis of 3D information recovered from video images and model views, respectively. During the sequence, motion parameters from video frames are used to compute interpolation parameters, and rendered model views are overlayed on corresponding video frames. Experimental results for real video frames and model views are given. Finally, discussion on the limitations of the method and subjects for future research are provided.

• Journal of Korea Multimedia Society
• /
• v.9 no.4
• /
• pp.421-433
• /
• 2006

This paper proposes a method to insert virtual objects into a real video stream based on feature tracking and camera pose estimation from a set of single-camera video frames. To insert or modify 3D shapes to target video frames, the transformation from the 3D objects to the projection of the objects onto the video frames should be revealed. It is shown that, without a camera calibration process, the 3D reconstruction is possible using multiple images from a single camera under the fixed internal camera parameters. The proposed approach is based on the simplification of the camera matrix of intrinsic parameters and the use of projective geometry. The method is particularly useful for augmented reality applications to insert or modify models to a real video stream. The proposed method is based on a linear parameter estimation approach for the auto-calibration step and it enhances the stability and reduces the execution time. Several experimental results are presented on real-world video streams, demonstrating the usefulness of our method for the augmented reality applications.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2021.10a
• /
• pp.521-523
• /
• 2021

Data augmentation are techniques used to increase the amount of data by using small amount of existing data. With the spread of the Internet, we can easily obtain data. However, there are still certain industries, like medicine, where it is difficult to obtain data. The same is true for image data in which a black screen is displayed on video wall controller. Because it is rare that a black screen is displayed during operation, it is not easy to obtain an image with a black screen. We propose a DCGAN based architecture that generate dataset using a small amount of black screen image.

• Journal of Korea Multimedia Society
• /
• v.14 no.2
• /
• pp.329-338
• /
• 2011

Recent augmentation reality demands more realistic multimedia data with the mixture of various media. High-technology for multimedia data which combines existing media data with various media such as audio and video dominates entire media industries. In particular, there is a growing need to serve augmentation reality, 3-dimensional contents and realtime interaction system development which are communication method and visualization tool in Internet. The existing services do not correspond to generate depth value for 3-dimensional space structure recovery which is to form solidity in existing contents. Therefore, it requires research for effective depth-map generation using 2-dimensional video. Complementing shortcomings of existing depth-map generation method using 2-dimensional video, this paper proposes an enhanced depth-map generation method that defines the depth direction in regard to loss location in a video in which none of existing algorithms has defined.

• The Journal of the Korea Contents Association
• /
• v.13 no.6
• /
• pp.1-8
• /
• 2013

It is generally required to use modeling or animation tools for inserting dynamic objects into augmented reality, and this process demands high expertise and complexity. This paper proposes a video object based authoring method that enables augmentation with dynamic video objects without such process. Integrated grab-cut and grow-cut method strips initial area of video target off the existing video clips, and snap-cut method is then applied to track objects' boundaries over frames so as to augment real world with continuous motion frames. Experiment shows video cut-out and authoring results achieved by only a few menu selections and simple correcting sketch.

• Proceedings of the IEEK Conference
• /
• 2006.06a
• /
• pp.685-686
• /
• 2006

Recently, various AR-based product design methodologies have been introduced. In this paper, we propose technologies for enhancing robust augmentation and immersive realization of virtual objects. A robust augmentation technology is developed for various lighting conditions and a partial solution is proposed for the hand occlusion problem that occurs when the virtual objects overlay the user' hands. It provides more immersive or natural images to the users. Finally, vibratory haptic cues by page motors as well as button clicking force feedback by modulating pneumatic pressures are proposed while interacting with virtual widgets. Also our system reduces gabs between modeling spaces and user spaces. An immersive game-phone model is selected to demonstrate that the users can control the direction of the car in the racing game by tilting a tangible object with the proposed augmented haptic and robust non-occluded visual feedback. The proposed methodologies will be contributed to the immersive realization of the conventional AR system.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.24 no.1
• /
• pp.49-54
• /
• 2024

Data imbalance is a common issue encountered in classification problems, stemming from a significant disparity in the number of samples between classes within the dataset. Such data imbalance typically leads to problems in classification models, including overfitting, underfitting, and misinterpretation of performance metrics. Methods to address this issue include resampling, augmentation, regularization techniques, and adjustment of loss functions. In this paper, we focus on loss function adjustment, particularly comparing the performance of various configurations of loss functions (Cross Entropy, Balanced Cross Entropy, two settings of Focal Loss: 𝛼 = 1 and 𝛼 = Balanced, Asymmetric Loss) on Multi-Class black-box video data with imbalance issues. The comparison is conducted using the I3D, and R3D_18 models.