• Journal of KIISE:Databases
• /
• v.29 no.3
• /
• pp.168-179
• /
• 2002

In this paper, we investigate video representation techniques that are the foundational work for the subsequent video processing such as video storage and retrieval. A video data set if a collection of video clips, each of which is a sequence of video frames and is represented by a multidimensional data sequence (MDS). An MDS is partitioned into video segments considering temporal relationship among frames, and then similar segments of the clip are grouped into video clusters. Thus, the video clip is represented by a small number of video clusters. The video segmentation and clustering algorithm, VDCluster, proposed in this paper guarantee clustering quality to south an extent that satisfies predefined conditions. The experiments show that our algorithm performs very effectively with respect to various video data sets.

• Proceedings of the Korea Institute of Convergence Signal Processing
• /
• 2006.06a
• /
• pp.45-48
• /
• 2006

This paper presents an approach to visual information based temporal indexing of video sequences. The objective of this work is the integration of an automatic face detection and a matching system for video indexing. The face detection is done using color information. The matching stage is based on the Principal Component Analysis (PCA) followed by the Minimax Probability Machine (MPM). Using PCA one feature vector is calculated for each face which is detected at the previous stage from the video sequence and MPM is applied to these feature vectors for matching with the training faces which are manually indexed after extracting from video sequences. The integration of the two stages gives good results. The rate of 86.3% correctly classified frames shows the efficiency of our system.

• Korean Journal of Geomatics
• /
• v.3 no.1
• /
• pp.23-32
• /
• 2003

Rigorous sensor and dynamic modeling techniques are required if spatial information is to be accurately extracted from video imagery. First, a mathematical model for an uncalibrated video camera and a description of a bundle adjustment with added parameters, for purposes of general block triangulation, is presented. This is followed by the application of invariance-based techniques, with constraints, to derive initial approximations for the camera parameters. Finally, dynamic modeling using the Kalman Filter is discussed. The results of various experiments with real video imagery, which apply the developed techniques, are given.

• Speech Sciences
• /
• v.12 no.1
• /
• pp.135-142
• /
• 2005

An important aspect of video indexing is the ability to segment video into meaningful segments, i.e., content-based video segmentation. Since the audio signal in the sound track is synchronized with image sequences in the video program, a speech signal in the sound track can be used to segment video into meaningful segments. In this paper, we propose a new approach to content-based video segmentation. This approach uses closed caption to construct a recognition network for speech recognition. Accurate time information for video segmentation is then obtained from the speech recognition process. For the video segmentation experiment for TV news programs, we made 56 video summaries successfully from 57 TV news stories. It demonstrates that the proposed scheme is very promising for content-based video segmentation.

• Journal of Multimedia Information System
• /
• v.8 no.3
• /
• pp.147-158
• /
• 2021

Versatile Video Coding (VVC) is the latest video coding standard developed by Joint Video Exploration Team (JVET). In VVC, the quadtree plus multi-type tree (QT+MTT) structure of coding unit (CU) partition is adopted, and its computational complexity is considerably high due to the brute-force search for recursive rate-distortion (RD) optimization. In this paper, we aim to reduce the time complexity of inter-picture prediction mode since the inter prediction accounts for a large portion of the total encoding time. The problem can be defined as classifying the split mode of each CU. To classify the split mode effectively, a novel convolutional neural network (CNN) called multi-level tree (MLT-CNN) architecture is introduced. For boosting classification performance, we utilize additional information including inter-picture information while training the CNN. The overall algorithm including the MLT-CNN inference process is implemented on VVC Test Model (VTM) 11.0. The CUs of size 128×128 can be the inputs of the CNN. The sequences are encoded at the random access (RA) configuration with five QP values {22, 27, 32, 37, 42}. The experimental results show that the proposed algorithm can reduce the computational complexity by 11.53% on average, and 26.14% for the maximum with an average 1.01% of the increase in Bjøntegaard delta bit rate (BDBR). Especially, the proposed method shows higher performance on the sequences of the A and B classes, reducing 9.81%~26.14% of encoding time with 0.95%~3.28% of the BDBR increase.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.41 no.5
• /
• pp.35-44
• /
• 2004

With increase in digitalmedia data, various video indexing and video sequence matching algorithms have been proposed to efficiently manage and utilize digital media. In this paper, we propose a robust video indexing algorithm to detect scene changes for video sequences with abrupt luminance variations and an efficient video sequence matching algorithm for video sequence query. To improve the accuracy and to reduce the computational complexity for video indexing with abrupt luminance variations, the proposed algorithm utilizes edge features as well as color features, which yields a remarkably better performance than conventional algorithms. In the proposed algorithm first we extract the candidate shot boundaries using color histograms and then determine using edge matching and luminance compensation if they are shot boundaries or luminance changes. If the scene contains trivial brighness variations, the edge matching and luminance compensation are performed only for shot boundaries. In experimental results, the proposed method gives remarkably a high performance and efficiency than the conventional methods with the similar computational complexity.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.9 no.1
• /
• pp.169-189
• /
• 2015

Detection of anomalous events from video streams is a challenging problem in many video surveillance applications. One such application that has received significant attention from the computer vision community is traffic video surveillance. In this paper, a Lossy Count based Sequential Temporal Pattern mining approach (LC-STP) is proposed for detecting spatio-temporal abnormal events (such as a traffic violation at junction) from sequences of video streams. The proposed approach relies mainly on spatial abstractions of each object, mining frequent temporal patterns in a sequence of video frames to form a regular temporal pattern. In order to detect each object in every frame, the input video is first pre-processed by applying Gaussian Mixture Models. After the detection of foreground objects, the tracking is carried out using block motion estimation by the three-step search method. The primitive events of the object are represented by assigning spatial and temporal symbols corresponding to their location and time information. These primitive events are analyzed to form a temporal pattern in a sequence of video frames, representing temporal relation between various object's primitive events. This is repeated for each window of sequences, and the support for temporal sequence is obtained based on LC-STP to discover regular patterns of normal events. Events deviating from these patterns are identified as anomalies. Unlike the traditional frequent item set mining methods, the proposed method generates maximal frequent patterns without candidate generation. Furthermore, experimental results show that the proposed method performs well and can detect video anomalies in real traffic video data.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.42 no.6
• /
• pp.27-32
• /
• 2005

An effective video watermarking algorithm is proposed to protect the copyright. The watermarking procedure is based on a three-dimensional discrete wavelet transform (3D DWT) and spread spectrum sequences. Two perceptual binary watermarks are preprocessed using mixing and pseudorandom permutation. After dividing the video sequence into video shots, the 3D DWT is performed, then the preprocessed watermarks are embedded into the 3D DWT coefficients, while considering robustness and invisibility, using two spread spectrum sequences defined as the user key. Experimental results show that the watermarked frames are subjectively indistinguishable from the original frames, plus the proposed video watermarking algorithm is sufficiently robust against such attacks as low pass filtering, frame dropping, frame average, and MPEG coding.

• Journal of Korea Multimedia Society
• /
• v.16 no.1
• /
• pp.42-49
• /
• 2013

The camera motion is accompanied with the translation and/or the rotation of objects in frames of a video sequence. An unnecessary rotation of objects declines the quality of the moving pictures and in addition is a primary cause of the viewers' fatigue. In this paper, a novel method for correcting rotated frames in video sequences is presented, where the modified Mojette transform is applied to the motion-compensated area in each frame. The Mojette transform is one of discrete Radon transforms, and is modified for correcting the rotated frames as follows. First, the bin values in the Mojette transform are determined by using pixels on the projection line and the interpolation of pixels adjacent to the line. Second, the bin values are calculated only at some area determined by the motion estimation between current and reference frames. Finally, only one bin at each projection is computed for reducing the amount of the calculation in the Mojette transform. Through the simulation carried out on various test video sequences, it is shown that the proposed scheme has good performance for correcting the rotation of frames in moving pictures.

• ETRI Journal
• /
• v.36 no.2
• /
• pp.224-231
• /
• 2014

In this paper, a method to extract temperature effect information using the color temperatures of video scenes with mapping to temperature effects is proposed to author temperature effects of multiple sensorial media content automatically. An authoring tool to apply the proposed method is also introduced. The temperature effects generated by the proposed method are evaluated by a subjective test to measure the level of satisfaction. The mean opinion score results show that most of the test video sequences receive an average of approximately four points (in a five-point scale), indicating that test video sequences (with the temperature effects generated by the proposed method) enhance levels of satisfaction.