• Journal of Korea Multimedia Society
• /
• v.20 no.10
• /
• pp.1628-1636
• /
• 2017

Such as closed caption, ancillary data, electronic program guide(EPG), data broadcasting, and etc, increased data for service convenience cause to degrade video quality of high definition contents. This article propose a method to transfer the closed caption data of video contents without video quality degradation. Video quality degradation does not cause in video compression by the block image insertion of caption data in DTV essential hidden area. Additionally the proposed methods have advantage to synchronize video, audio, and caption from preinserted script without time delay.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2002.11a
• /
• pp.619-623
• /
• 2002

We can speak caption as information that can not except caption on sports video. The sports highlight were composed that we recognize captioning. This paper is the necessary work to the middle-step to analysis the caption through the retrieval and discrimination from the position of caption. This paper improve at first and simplify the image through the excellent threshold value algorithm in the preprocessing and then use method that can analysis caption through the multiplex frame merging algorithm. Its speed performing shows up higher and simplier than the region growing process.

• Journal of Broadcast Engineering
• /
• v.5 no.1
• /
• pp.113-122
• /
• 2000

It is sometimes necessary to remove the captions and recover original images from video images already broadcast, When the number of images requiring such recovery is small, manual processing is possible, but as the number grows it would be very difficult to do it manually. Therefore, a method for recovering original image for the caption areas in needed. Traditional research on image restoration has focused on restoring blurred images to sharp images using frequency filtering or video coding for transferring video images. This paper proposes a method for automatically recovering original image using BMA(Block Matching Algorithm). We extract information on caption regions and scene change that is used as a prior-knowledge for recovering original image. From the result of caption information detection, we know the start and end frames of captions in video and the character areas in the caption regions. The direction for the recovery is decided using information on the scene change and caption region(the start and end frame for captions). According to the direction, we recover the original image by performing block matching for character components in extracted caption region. Experimental results show that the case of stationary images with little camera or object motion is well recovered. We see that the case of images with motion in complex background is also recovered.

• KIPS Transactions on Software and Data Engineering
• /
• v.6 no.4
• /
• pp.203-210
• /
• 2017

In this paper, we propose an effective neural network model for image caption generation and model transfer. This model is a kind of multi-modal recurrent neural network models. It consists of five distinct layers: a convolution neural network layer for extracting visual information from images, an embedding layer for converting each word into a low dimensional feature, a recurrent neural network layer for learning caption sentence structure, and a multi-modal layer for combining visual and language information. In this model, the recurrent neural network layer is constructed by LSTM units, which are well known to be effective for learning and transferring sequence patterns. Moreover, this model has a unique structure in which the output of the convolution neural network layer is linked not only to the input of the initial state of the recurrent neural network layer but also to the input of the multimodal layer, in order to make use of visual information extracted from the image at each recurrent step for generating the corresponding textual caption. Through various comparative experiments using open data sets such as Flickr8k, Flickr30k, and MSCOCO, we demonstrated the proposed multimodal recurrent neural network model has high performance in terms of caption accuracy and model transfer effect.

• Journal of the Korea Computer Industry Society
• /
• v.3 no.7
• /
• pp.901-914
• /
• 2002

In this paper, We propose an efficient automatic caption detection and location method, caption recognition using FE-MCBP(Feature Extraction based Multichained BackPropagation) neural network for content based retrieval of video. Frames are selected at fixed time interval from video and key frames are selected by gray scale histogram method. for each key frames, segmentation is performed and caption lines are detected using line scan method. lastly each characters are separated. This research improves speed and efficiency by color segmentation using local maximum analysis method before line scanning. Caption detection is a first stage of multimedia database organization and detected captions are used as input of text recognition system. Recognized captions can be searched by content based retrieval method.

• Journal of Digital Contents Society
• /
• v.18 no.7
• /
• pp.1323-1331
• /
• 2017

Despite the fact that the copyright has grown into a large-scale business, there are many constant problems especially in image copyright. In this study, we propose an automatic object extraction and classification system for the scanned educational book image by combining document image processing and intelligent information technology like deep learning. First, the proposed technology removes noise component and then performs a visual attention assessment-based region separation. Then we carry out grouping operation based on extracted block areas and categorize each block as a picture or a character area. Finally, the caption area is extracted by searching around the classified picture area. As a result of the performance evaluation, it can be seen an average accuracy of 83% in the extraction of the image and caption area. For only image region detection, up-to 97% of accuracy is verified.

• The Korean Journal of Applied Statistics
• /
• v.34 no.2
• /
• pp.191-204
• /
• 2021

Visual Question Answering (VQA) and image captioning are tasks that require understanding of the features of images and linguistic features of text. Therefore, co-attention may be the key to both tasks, which can connect image and text. In this paper, we propose a model to achieve high performance for VQA by image caption generated using a pretrained standard transformer model based on MSCOCO dataset. Captions unrelated to the question can rather interfere with answering, so some captions similar to the question were selected to use based on a similarity to the question. In addition, stopwords in the caption could not affect or interfere with answering, so the experiment was conducted after removing stopwords. Experiments were conducted on VQA-v2 data to compare the proposed model with the deep modular co-attention network (MCAN) model, which showed good performance by using co-attention between images and text. As a result, the proposed model outperformed the MCAN model.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.41 no.3
• /
• pp.1-8
• /
• 2004

In this paper, we propose a method for restoring trademark and caption areas using an isophote. In our method, the image restoration problem is modeled as an optimization problem, which in our case, is solved by a cost function with isophote constraint that is minimized using a GA The technique creates an optimal connection of all pairs of isophotes disconnected by a caption in the frame. For connecting the disconnected isophotes, we estimate the value of the smoothness, given by the best chromosomes of the GA and project this value in the isophote direction. Experimental results show that the isophote operator worked better than Laplacian operator for image restoration, and the proposed method has a great possibility for automatic restoration of a region in an advertisement scene.

• Speech Sciences
• /
• v.12 no.1
• /
• pp.135-142
• /
• 2005

An important aspect of video indexing is the ability to segment video into meaningful segments, i.e., content-based video segmentation. Since the audio signal in the sound track is synchronized with image sequences in the video program, a speech signal in the sound track can be used to segment video into meaningful segments. In this paper, we propose a new approach to content-based video segmentation. This approach uses closed caption to construct a recognition network for speech recognition. Accurate time information for video segmentation is then obtained from the speech recognition process. For the video segmentation experiment for TV news programs, we made 56 video summaries successfully from 57 TV news stories. It demonstrates that the proposed scheme is very promising for content-based video segmentation.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.8 no.3
• /
• pp.630-636
• /
• 2004

According to the increase of audio and video data utilization, the presentation of multimedia data contents and the work of retrieving, storing and manipulating a multimedia data have been the focus of recent work. The display for multimedia data should retrieve and access the contents easily that users want to present. This study is about the design and implementation of a system to retrieve multimedia data based on the contents of documentation or the caption information of a multimedia data for retrieving documentation including multimedia data. It intends to develop an filtering step to retrieve all of keyword within the caption information of multimedia data and text of a documentation. Also, the system is designed to retrieve a large amount of data quickly using an inverted file structure available for B+ tree.