• Journal of Intelligence and Information Systems
• /
• v.17 no.3
• /
• pp.203-219
• /
• 2011

Previously common users just want to watch the video contents without any specific requirements or purposes. However, in today's life while watching video user attempts to know and discover more about things that appear on the video. Therefore, the requirements for finding multimedia or browsing information of objects that users want, are spreading with the increasing use of multimedia such as videos which are not only available on the internet-capable devices such as computers but also on smart TV and smart phone. In order to meet the users. requirements, labor-intensive annotation of objects in video contents is inevitable. For this reason, many researchers have actively studied about methods of annotating the object that appear on the video. In keyword-based annotation related information of the object that appeared on the video content is immediately added and annotation data including all related information about the object must be individually managed. Users will have to directly input all related information to the object. Consequently, when a user browses for information that related to the object, user can only find and get limited resources that solely exists in annotated data. Also, in order to place annotation for objects user's huge workload is required. To cope with reducing user's workload and to minimize the work involved in annotation, in existing object-based annotation automatic annotation is being attempted using computer vision techniques like object detection, recognition and tracking. By using such computer vision techniques a wide variety of objects that appears on the video content must be all detected and recognized. But until now it is still a problem facing some difficulties which have to deal with automated annotation. To overcome these difficulties, we propose a system which consists of two modules. The first module is the annotation module that enables many annotators to collaboratively annotate the objects in the video content in order to access the semantic data using Linked Data. Annotation data managed by annotation server is represented using ontology so that the information can easily be shared and extended. Since annotation data does not include all the relevant information of the object, existing objects in Linked Data and objects that appear in the video content simply connect with each other to get all the related information of the object. In other words, annotation data which contains only URI and metadata like position, time and size are stored on the annotation sever. So when user needs other related information about the object, all of that information is retrieved from Linked Data through its relevant URI. The second module enables viewers to browse interesting information about the object using annotation data which is collaboratively generated by many users while watching video. With this system, through simple user interaction the query is automatically generated and all the related information is retrieved from Linked Data and finally all the additional information of the object is offered to the user. With this study, in the future of Semantic Web environment our proposed system is expected to establish a better video content service environment by offering users relevant information about the objects that appear on the screen of any internet-capable devices such as PC, smart TV or smart phone.

• The Journal of Korea Robotics Society
• /
• v.14 no.2
• /
• pp.139-149
• /
• 2019

In this paper, we present auto-annotation tool and synthetic dataset using 3D CAD model for deep learning based object detection. To be used as training data for deep learning methods, class, segmentation, bounding-box, contour, and pose annotations of the object are needed. We propose an automated annotation tool and synthetic image generation. Our resulting synthetic dataset reflects occlusion between objects and applicable for both underwater and in-air environments. To verify our synthetic dataset, we use MASK R-CNN as a state-of-the-art method among object detection model using deep learning. For experiment, we make the experimental environment reflecting the actual underwater environment. We show that object detection model trained via our dataset show significantly accurate results and robustness for the underwater environment. Lastly, we verify that our synthetic dataset is suitable for deep learning model for the underwater environments.

• Journal of Intelligence and Information Systems
• /
• v.21 no.1
• /
• pp.83-98
• /
• 2015

Many TV viewers use mainly portal sites in order to retrieve information related to broadcast while watching TV. However retrieving information that people wanted needs a lot of time to retrieve the information because current internet presents too much information which is not required. Consequentially, this process can't satisfy users who want to consume information immediately. Interactive video is being actively investigated to solve this problem. An interactive video provides clickable objects, areas or hotspots to interact with users. When users click object on the interactive video, they can see additional information, related to video, instantly. The following shows the three basic procedures to make an interactive video using interactive video authoring tool: (1) Create an augmented object; (2) Set an object's area and time to be displayed on the video; (3) Set an interactive action which is related to pages or hyperlink; However users who use existing authoring tools such as Popcorn Maker and Zentrick spend a lot of time in step (2). If users use wireWAX then they can save sufficient time to set object's location and time to be displayed because wireWAX uses vision based annotation method. But they need to wait for time to detect and track object. Therefore, it is required to reduce the process time in step (2) using benefits of manual annotation method and vision-based annotation method effectively. This paper proposes a novel annotation method allows annotator to easily annotate based on face area. For proposing new annotation method, this paper presents two steps: pre-processing step and annotation step. The pre-processing is necessary because system detects shots for users who want to find contents of video easily. Pre-processing step is as follow: 1) Extract shots using color histogram based shot boundary detection method from frames of video; 2) Make shot clusters using similarities of shots and aligns as shot sequences; and 3) Detect and track faces from all shots of shot sequence metadata and save into the shot sequence metadata with each shot. After pre-processing, user can annotates object as follow: 1) Annotator selects a shot sequence, and then selects keyframe of shot in the shot sequence; 2) Annotator annotates objects on the relative position of the actor's face on the selected keyframe. Then same objects will be annotated automatically until the end of shot sequence which has detected face area; and 3) User assigns additional information to the annotated object. In addition, this paper designs the feedback model in order to compensate the defects which are wrong aligned shots, wrong detected faces problem and inaccurate location problem might occur after object annotation. Furthermore, users can use interpolation method to interpolate position of objects which is deleted by feedback. After feedback user can save annotated object data to the interactive object metadata. Finally, this paper shows interactive video authoring system implemented for verifying performance of proposed annotation method which uses presented models. In the experiment presents analysis of object annotation time, and user evaluation. First, result of object annotation average time shows our proposed tool is 2 times faster than existing authoring tools for object annotation. Sometimes, annotation time of proposed tool took longer than existing authoring tools, because wrong shots are detected in the pre-processing. The usefulness and convenience of the system were measured through the user evaluation which was aimed at users who have experienced in interactive video authoring system. Recruited 19 experts evaluates of 11 questions which is out of CSUQ(Computer System Usability Questionnaire). CSUQ is designed by IBM for evaluating system. Through the user evaluation, showed that proposed tool is useful for authoring interactive video than about 10% of the other interactive video authoring systems.

• Journal of Broadcast Engineering
• /
• v.25 no.3
• /
• pp.386-398
• /
• 2020

We developed an integrative annotation program that can perform data labeling process for deep learning models in object recognition. The program utilizes the basic GUI library of Python and configures crawler functions that allow data collection in real time. Retinanet was used to implement an automatic annotation function. In addition, different data labeling formats for Pascal-VOC, YOLO and Retinanet were generated. Through the experiment of the proposed method, a domestic vehicle image dataset was built, and it is applied to Retinanet and YOLO as the training and test set. The proposed system classified the vehicle model with the accuracy of about 94%.

• Journal of Korea Multimedia Society
• /
• v.23 no.8
• /
• pp.986-1005
• /
• 2020

In this paper, we propose AnoVid, an automated video annotation tool based on deep neural networks, that automatically generates various meta data for each scene or shot in a long drama video containing rich elements. To this end, a novel meta data schema for drama video is designed. Based on this schema, the AnoVid video annotation tool has a total of six deep neural network models for object detection, place recognition, time zone recognition, person recognition, activity detection, and description generation. Using these models, the AnoVid can generate rich video annotation data. In addition, AnoVid provides not only the ability to automatically generate a JSON-type video annotation data file, but also provides various visualization facilities to check the video content analysis results. Through experiments using a real drama video, "Misaeing", we show the practical effectiveness and performance of the proposed video annotation tool, AnoVid.

• Journal of the Korea Society of Computer and Information
• /
• v.3 no.1
• /
• pp.25-34
• /
• 1998

There are two major method for expressing parallelim in functional languages. The one is the strictness analysis and the other the annotation. The strictness analysis is a method that a compiler detects parallelism and expresses the detected information in the object program. The annotation is a method that a programmer detects parallelism and expresses in the source program. This study is on the annotation and is aimed at construction of a translator for a functional language which has an annotation syntax. The translator translates a source program to enriched lambda-calculus graphs. The translator is implemented in C using compiler development tools such as YACC and Lex, under UNIX environments. In this paper we present the design and implementation techniques for developing the front-end of the translator.

• Journal of the Korea Society of Computer and Information
• /
• v.13 no.5
• /
• pp.11-18
• /
• 2008

One of important Issues in computer graphics field is to communicate among users in virtual world like secondlife. However, in 3D virtual world, users' needs to wish to build their own contents in 3D virtual space are rising, similarly, users produce own homepage and animation, and leave writing in notice board. In this paper, we tried to achieve this by introducing semantic annotation object concept, which is a kind of annotation method in 3D virtual world. User can retrieve an 3D object by searching corresponding annotation data. This method can build semantic 3D virtual world and enable users to search 3D objects by integrating 3D object and 2D semantic multimedia information. Also, through a comparison experiment with proposal system and general 3D virtual world. the performance of proposed system is evaluated.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.18 no.6
• /
• pp.1109-1116
• /
• 2023

Object tracking is used to track a goal in a video sequence by using coordinate information provided as annotation in the first frame of the video. In this paper, we propose a tracking algorithm that combines deep features and region inference modules to improve object tracking accuracy. In order to obtain sufficient object information, a convolution neural network was designed with a Siamese network structure. For object region inference, the region proposal network and overlapping confidence module were applied and used for tracking. The performance of the proposed tracking algorithm was evaluated using the Object Tracking Benchmark dataset, and it achieved 69.1% in the Success index and 89.3% in the Precision Metrics.

• Smart Media Journal
• /
• v.11 no.4
• /
• pp.38-45
• /
• 2022

Object detection models have become the current tool of choice for plant disease detection in precision agriculture. Most existing research improves the performance by ameliorating networks and optimizing the loss function. However, the data-centric part of a whole project also needs more investigation. In this paper, we proposed a systematic strategy with three different annotation methods for plant disease detection: local, semi-global, and global label. Experimental results on our paprika disease dataset show that a single class annotation with semi-global boxes may improve accuracy. In addition, we also studied the noise factor during the labeling process. An ablation study shows that annotation noise within 10% is acceptable for keeping good performance. Overall, this data-centric numerical analysis helps us to understand the significance of annotation methods, which provides practitioners a way to obtain higher performance and reduce annotation costs on plant disease detection tasks. Our work encourages researchers to pay more attention to label quality and the essential issues of labeling methods.

• Journal of Korea Multimedia Society
• /
• v.7 no.9
• /
• pp.1321-1328
• /
• 2004

In this article, we present a new DICOM object able to create image-based annotations in DICOM image. Since the proposed image-based annotation uses images themselves of annotation, various types like character, sketch, and scanning image, etc., can be imported into annotation easily. The proposed annotation is inserted into DICOM image directly but they do not influence original DICOM image quality by using independent data channel. The proposed annotation is expected to be very useful to medium and small clinics that cannot afford picture archiving and communication systems or electronic medical record.