• Journal of Information Processing Systems
• /
• v.14 no.6
• /
• pp.1318-1330
• /
• 2018

Video captioning refers to the process of extracting features from a video and generating video captions using the extracted features. This paper introduces a deep neural network model and its learning method for effective video captioning. In this study, visual features as well as semantic features, which effectively express the video, are also used. The visual features of the video are extracted using convolutional neural networks, such as C3D and ResNet, while the semantic features are extracted using a semantic feature extraction network proposed in this paper. Further, an attention-based caption generation network is proposed for effective generation of video captions using the extracted features. The performance and effectiveness of the proposed model is verified through various experiments using two large-scale video benchmarks such as the Microsoft Video Description (MSVD) and the Microsoft Research Video-To-Text (MSR-VTT).

• Proceedings of the Optical Society of Korea Conference
• /
• 1996.09a
• /
• pp.21-21
• /
• 1996

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.43 no.2 s.308
• /
• pp.57-64
• /
• 2006

Tn this paper, We propose a user friendly object-based multimedia retrieval system using the HCNN(HippoCampus Neural Network. Most existing approaches to content-based retrieval rely on query by example or user based low-level features such as color, shape, texture. In this paper we perform a scene change detection and key frame extraction for the compressed video stream that is video compression standard such as MPEG. We propose a method for automatic color object extraction and ACE(Adaptive Circular filter and Edge) of content-based multimedia retrieval system. And we compose multimedia retrieval system after learned by the HCNN such extracted features. Proposed HCNN makes an adaptive real-time content-based multimedia retrieval system using excitatory teaming method that forwards important features to long-term memories and inhibitory learning method that forwards unimportant features to short-term memories controlled by impression.

• Journal of the Institute of Convergence Signal Processing
• /
• v.24 no.4
• /
• pp.221-228
• /
• 2023

Physical and mental changes caused by emotions can affect various behaviors, such as driving or learning behavior. Therefore, recognizing these emotions is a very important task because it can be used in various industries, such as recognizing and controlling dangerous emotions while driving. In this paper, we attempted to solve the emotion recognition task by implementing a multimodal model that recognizes emotions using both audio and video data from different domains. After extracting voice from video data using RAVDESS data, features of voice data are extracted through a model using 2D-CNN. In addition, the video data features are extracted using a slowfast feature extractor. And the information contained in the audio and video data, which have different domains, are combined into one feature that contains all the information. Afterwards, emotion recognition is performed using the combined features. Lastly, we evaluate the conventional methods that how to combine results from models and how to vote two model's results and a method of unifying the domain through feature extraction, then combining the features and performing classification using a classifier.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.48 no.3
• /
• pp.46-53
• /
• 2011

This paper proposes a SURF (Speeded Up Robust Features) based face recognition method which is one of typical interest point extraction algorithms. In general, SURF based object recognition is performed in interest point extraction and matching. In this paper, although, proposed method is employed not only in interest point extraction and matching, but also in face image rotation and interest point verification. image rotation is performed to increase the number of interest points and interest point verification is performed to find interest points which were matched correctly. Although proposed SURF based face recognition method requires more computation time than PCA based one, it shows better recognition rate than PCA algorithm. Through this experimental result, I confirmed that interest point extraction algorithm also can be adopted in face recognition.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.19 no.5
• /
• pp.629-634
• /
• 2009

In real-time system application, the feature extraction and scoring algorithm for classification of the passive sonar target has the following problems: it requires an accurate and efficient feature extraction method because it is very difficult to distinguish the features of the propeller shaft rate (PSR) and the blade rate (BR) from the frequency spectrum in real-time, it requires a robust and effective feature scoring method because the classification database (DB) composed of extracted features is noised and incomplete, and further, it requires an easy design procedure in terms of structures and parameters. To solve these problems, an intelligent feature extraction and scoring algorithm using the evolution strategy (ES) and the fuzzy theory is proposed here. To verify the performance of the proposed algorithm, a passive sonar target classification is performed in real-time. Simulation results show that the proposed algorithm effectively solves sonar classification problems in real-time.

• International Journal of Control, Automation, and Systems
• /
• v.2 no.1
• /
• pp.125-133
• /
• 2004

In this paper, we present a new method that combines the characteristics of edge in-formation and second-order neural networks for the classification of structural textures. The edges of a texture are extracted using an edge detection approach. From this edge information, classification features called second-order features are obtained. These features are fed into a second-order neural network for training and subsequent classification. It will be shown that the main disadvantage of using structural methods in texture classifications, namely, the difficulty of the extraction of texels, is overcome by the proposed method.

• Proceedings of the IEEK Conference
• /
• 2001.06c
• /
• pp.77-80
• /
• 2001

In general, the informations of the inner image that user interested in are limited to a special domain. In this paper, as using Wavelet Transform for dividing image into high frequency and low frequency, We can separate foreground including many data. After calculating object boundary of separated part, We extract special features using Color Coherence Vector. According to results of this experiment, the method of comparing data extracting foreground features is more effective than comparing data extracting features of entire image when we extract the image user interested in.

• Proceedings of the IEEK Conference
• /
• 2003.11a
• /
• pp.453-456
• /
• 2003

Compared with other features of the image, color features are less sensitive to noise and background complication. Besides, this adding to object segmentation has more accuracy of image retrieval. This paper presents object segmentation and HAQ(Histogram Analysis and Quantization) algorithm approach to extract features(the object information and the characteristic colors) of an image. The empirical results shows that this method presents exactly spatial and color information of an image as image retrieval's feature.

• Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
• /
• v.33 no.5
• /
• pp.363-371
• /
• 2015

Feature extraction is one of the main goals in many remote sensing analyses. After high-resolution imagery became more available, it became possible to extract more detailed and specific features. Thus, considerable image segmentation algorithms have been developed, because traditional pixel-based analysis proved insufficient for high-resolution imagery due to its inability to handle the internal variability of complex scenes. However, the individual segmentation method, which simply uses color layers, is limited in its ability to extract various target features with different spectral and shape characteristics. Spectral indices can be used to support effective feature extraction by helping to identify abundant surface materials. This study aims to evaluate a feature extraction method based on a segmentation technique with spectral indices. We tested the extraction of diverse target features-such as buildings, vegetation, water, and shadows from eight band WorldView-2 satellite image using decision tree classification and used the result to draw the appropriate spectral indices for each specific feature extraction. From the results, We identified that spectral band ratios can be applied to distinguish feature classes simply and effectively.