• Journal of Korea Multimedia Society
• /
• v.25 no.2
• /
• pp.247-256
• /
• 2022

Visual tracking of selected target objects is fundamental challenging problems in computer vision. Object tracking localize the region of target object with bounding box in the video. We propose a Siam-FPN based custom fully CNN to solve visual tracking problems by regressing the target area in an end-to-end manner. A method of preserving the feature information flow using a feature map connection structure was applied. In this way, information is preserved and emphasized across the network. To regress object region and to classify object, the region proposal network was connected with the Siamese network. The performance of the tracking algorithm was evaluated using the OTB-100 dataset. Success Plot and Precision Plot were used as evaluation matrix. As a result of the experiment, 0.621 in Success Plot and 0.838 in Precision Plot were achieved.

• Journal of Information Processing Systems
• /
• v.14 no.6
• /
• pp.1318-1330
• /
• 2018

Video captioning refers to the process of extracting features from a video and generating video captions using the extracted features. This paper introduces a deep neural network model and its learning method for effective video captioning. In this study, visual features as well as semantic features, which effectively express the video, are also used. The visual features of the video are extracted using convolutional neural networks, such as C3D and ResNet, while the semantic features are extracted using a semantic feature extraction network proposed in this paper. Further, an attention-based caption generation network is proposed for effective generation of video captions using the extracted features. The performance and effectiveness of the proposed model is verified through various experiments using two large-scale video benchmarks such as the Microsoft Video Description (MSVD) and the Microsoft Research Video-To-Text (MSR-VTT).

• International journal of advanced smart convergence
• /
• v.11 no.2
• /
• pp.88-94
• /
• 2022

The relationship between sound and vision was experimentally confirmed by physicist Ernst Florens Friedrich Chladni as early as the 18th century and formally entered into systematic research. With the development of emerging media technology, sound reactive type visual content is not limited to a single visual interaction based on the vibration of sound, and its visual content shows a diversified and scalable development trend according to different purposes in many fields. This study analyzes the development and changes of sound visual art contents from early stage to modernization, and analyzes the development characteristic of sound visual art content in different fields and scene environments influence by interactive media, new media technologies and devices by means of case analysis. Through this research, it is expected that the sound reactive type visual art content can continue to develop and extend in the existing fields, while explore the scalability of the application of sound reactive type visual art content in more fields.

• Korean Journal of Remote Sensing
• /
• v.36 no.4
• /
• pp.545-555
• /
• 2020

This paper focuses on automatic image registration between EO (Electro-Optical) and IR (InfraRed) satellite images with different spectral properties using block-based approach and simple preprocessing technique to enhance the performance of feature matching. If unpreprocessed EO and IR images from Kompsat-3A satellite were applied to local feature matching algorithms(Scale Invariant Feature Transform, Speed-Up Robust Feature, etc.), image registration algorithm generally failed because of few detected feature points or mismatched pairs despite of many detected feature points. In this paper, we proposed a new image registration method which improved the performance of feature matching with block-based registration process on 9-divided image and pre-processing technique based on adaptive histogram equalization. The proposed method showed better performance than without our proposed technique on visual inspection and I-RMSE. This study can be used for automatic image registration between various images acquired from different sensors.

• IEIE Transactions on Smart Processing and Computing
• /
• v.4 no.2
• /
• pp.110-114
• /
• 2015

This paper reports a novel object-detection technique in video sequences. The proposed algorithm consists of detection of objects in illuminated and occluded videos by using object features and a neural network technique. It consists of two functional modules: region-based object feature extraction and continuous detection of objects in video sequences with region features. This scheme is proposed as an enhancement of the Lowe's scale-invariant feature transform (SIFT) object detection method. This technique solved the high computation time problem of feature generation in the SIFT method. The improvement is achieved by region-based feature classification in the objects to be detected; optimal neural network-based feature reduction is presented in order to reduce the object region feature dataset with winner pixel estimation between the video frames of the video sequence. Simulation results show that the proposed scheme achieves better overall performance than other object detection techniques, and region-based feature detection is faster in comparison to other recent techniques.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.53 no.3
• /
• pp.107-113
• /
• 2016

Visual discomfort is a common problem in three-dimensional videos. Among the methods to overcome visual discomfort presented in current research, disparity adjustment methods provide little guidance in determining the condition for disparity control. We propose a diaprity adjustment based on the characteristics of disparity distribution on visual comfort, where the visual comfort level is used as the adjustment paramter, in parallax barrier type auto-stereoscopic 3D display. In this paper, we use the horizontal image shift method for disparity adjustment to enhance visual comfort. The speeded-up robust feature is used to estimate the disparity distribution of 3D sequences, and the required amount for disparity control is chosen based on the pre-defined characteristics of disparity distribution on visual comfort. To evaluate the performance of the proposed method, we used a 3D equipment. Subjective tests were conducted at the fixed optimal viewing distance. The results show that comfortable videos were generated based on the proposed disparity adjustment method.

• Proceedings of the Korean Society of Precision Engineering Conference
• /
• 2013.05a
• /
• pp.1443-1444
• /
• 2013

• Journal of the Korean Institute of Telematics and Electronics T
• /
• v.36T no.2
• /
• pp.48-55
• /
• 1999

To cope actively with the changes of external environments, it is necessary that a robot should have visual feedback control (VFC) using image informations. A VFC system consists of a manipulator and camera(s). For the fixed visual system, when feature value are located at the same line, we have a problem of singular value unable to be controlled by VFC. As a solution, we may define state values of the image Jacobians, then, by making comparisons and evaluations of feature values, select available ones. This method, however, has a demerit increasing numbers of feature values. To solve the problem, moving cameras of VFC system actively, we suggest an algorithm which dose not cause singular value, and prove its availability through simulations.

• Journal of the Institute of Convergence Signal Processing
• /
• v.11 no.2
• /
• pp.143-150
• /
• 2010

Markerless augmented reality system must have the capability to recognize and match natural objects both in indoor and outdoor environment. In this paper, a novel approach is proposed for extracting features and recognizing natural objects using visual descriptors and codebooks. Since the augmented reality applications are sensitive to speed of operation and real time performance, our work mainly focused on recognition of multi-class natural objects and reduce the computing time for classification and feature extraction. SIFT(scale invariant feature transforms) and SURF(speeded up robust feature) are used to extract features from natural objects during training and testing, and their performance is compared. Then we form visual codebook from the high dimensional feature vectors using clustering algorithm and recognize the objects using naive Bayes classifier.

• Journal of Institute of Control, Robotics and Systems
• /
• v.21 no.12
• /
• pp.1109-1114
• /
• 2015

This paper presents an improved learning-based visual inspection method for auto parts inspection in severe lighting changes. Automobile sunroof frames are produced automatically by robots in most production lines. In the sunroof frame manufacturing process, there is a quality problem with some parts such as volts are missed. Instead of manual sampling inspection using some mechanical jig instruments, a learning-based machine vision system was proposed in the previous research[1]. But, in applying the actual sunroof frame production process, the inspection accuracy of the proposed vision system is much lowered because of severe illumination changes. In order to overcome this capricious environment, some selective feature vectors and cascade classifiers are used for each auto parts. And we are able to improve the inspection accuracy through the re-learning concept for the misclassified data. The effectiveness of the proposed visual inspection method is verified through sufficient experiments in a real sunroof production line.