• Journal of Intelligence and Information Systems
• /
• v.28 no.1
• /
• pp.89-106
• /
• 2022

Recently, the amount of video data collected from smartphones, CCTVs, black boxes, and high-definition cameras has increased rapidly. According to the increasing video data, the requirements for analysis and utilization are increasing. Due to the lack of skilled manpower to analyze videos in many industries, machine learning and artificial intelligence are actively used to assist manpower. In this situation, the demand for various computer vision technologies such as object detection and tracking, action detection, emotion detection, and Re-ID also increased rapidly. However, the object detection and tracking technology has many difficulties that degrade performance, such as re-appearance after the object's departure from the video recording location, and occlusion. Accordingly, action and emotion detection models based on object detection and tracking models also have difficulties in extracting data for each object. In addition, deep learning architectures consist of various models suffer from performance degradation due to bottlenects and lack of optimization. In this study, we propose an video analysis system consists of YOLOv5 based DeepSORT object tracking model, SlowFast based action recognition model, Torchreid based Re-ID model, and AWS Rekognition which is emotion recognition service. Proposed model uses single-linkage hierarchical clustering based Re-ID and some processing method which maximize hardware throughput. It has higher accuracy than the performance of the re-identification model using simple metrics, near real-time processing performance, and prevents tracking failure due to object departure and re-emergence, occlusion, etc. By continuously linking the action and facial emotion detection results of each object to the same object, it is possible to efficiently analyze videos. The re-identification model extracts a feature vector from the bounding box of object image detected by the object tracking model for each frame, and applies the single-linkage hierarchical clustering from the past frame using the extracted feature vectors to identify the same object that failed to track. Through the above process, it is possible to re-track the same object that has failed to tracking in the case of re-appearance or occlusion after leaving the video location. As a result, action and facial emotion detection results of the newly recognized object due to the tracking fails can be linked to those of the object that appeared in the past. On the other hand, as a way to improve processing performance, we introduce Bounding Box Queue by Object and Feature Queue method that can reduce RAM memory requirements while maximizing GPU memory throughput. Also we introduce the IoF(Intersection over Face) algorithm that allows facial emotion recognized through AWS Rekognition to be linked with object tracking information. The academic significance of this study is that the two-stage re-identification model can have real-time performance even in a high-cost environment that performs action and facial emotion detection according to processing techniques without reducing the accuracy by using simple metrics to achieve real-time performance. The practical implication of this study is that in various industrial fields that require action and facial emotion detection but have many difficulties due to the fails in object tracking can analyze videos effectively through proposed model. Proposed model which has high accuracy of retrace and processing performance can be used in various fields such as intelligent monitoring, observation services and behavioral or psychological analysis services where the integration of tracking information and extracted metadata creates greate industrial and business value. In the future, in order to measure the object tracking performance more precisely, there is a need to conduct an experiment using the MOT Challenge dataset, which is data used by many international conferences. We will investigate the problem that the IoF algorithm cannot solve to develop an additional complementary algorithm. In addition, we plan to conduct additional research to apply this model to various fields' dataset related to intelligent video analysis.

• Proceedings of the Korean Institute of Navigation and Port Research Conference
• /
• 2003.05a
• /
• pp.181-186
• /
• 2003

The satellite navigation system is widely used for identifying a user's position regardless of weather or geographic conditions and also make effect on new technology of marine LBS(Location Based Service), which has the technology of geographic information such as the ENC. Generally, there are conceivable systems of marine LBS such as ECDIS, or ECS that use the ENC itself with powerful processor in installed type on ships bridge. Since the ENC is relatively heavy structure with dummy format for data transfer between different systems, we should reduce the ENC to small and compact size in order to use it in mobile platform. In this paper, we assumed that the mobile system like PDA, or Webpad can be used for small capability of mobile platform. However, the ENC should be updated periodically by update profile data produced by HO. If we would reduce the ENC without a consideration of update, we could not get newly updated data furthermore. As summary, we studied considerations for ENC reduction with update capability. It will make the ENC be useful in many mobile platforms for various applications.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.43 no.6 s.312
• /
• pp.26-33
• /
• 2006

Film restoration is to detect the location and extent of defected regions from a given movie film, and if present, to reconstruct the lost information of each region. It has gained increasing attention by many researchers, to support multimedia service of high quality. In general, an old film is degraded by dust, scratch, flick, and so on. Among these, the most frequent degradation is the scratch. So far techniques for the scratch restoration have been developed, but they have limited applicability when dealing with all kinds of scratches. To fully support the automatic scratch restoration, the system should be developed that can detect all kinds of scratches from a given frame of old films. This paper presents a neurual network (NN)-based texture classifier that automatically detect all kinds of scratches from frames in old films. To facilitate the detection of various scratch sizes, we use a pyramid of images generated from original frames by having the resolution at three levels. The image at each level is scanned by the NN-based classifier, which divides the input image into scratch regions and non-scratch regions. Then, to reduce the computational cost, the NN-based classifier is only applied to the edge pixels. To assess the validity of the proposed method, the experiments have been performed on old films and animations with all kinds of scratches, then the results show the effectiveness of the proposed method.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.19 no.1
• /
• pp.622-632
• /
• 2018

With the recent changes in the social environment, the growth in the so-called 'five major crimes' has been highlighted as one of the causes of anxiety in Koreans' lives. Many attempts have been made to solve this problem; however, it is still difficult to secure the location information of the socially vulnerable in emergency situations and to precisely identify the features and clothing of criminals and track them using current image analysis technology. Therefore, the development of precision positioning technology and support services along with intelligent security service technology based on spatial information has been given a high priority. This study suggested measures that could be continuously applied to link technologies and services with high linkability by analyzing technologies based on spatial information and other fields. To establish measures for linkage between intelligent security technology and other technologies and services, this study analyzed the existing technologies and research trends in intelligent security technology, and reviewed linkable services according to five criteria established to evaluate their linkability. Based on this analysis, three technologies with high linkability were ultimately selected, and measures for linkage were established. It is expected that the linkage measures derived using the objective evaluation criteria will serve as a stepping stone for promoting active technology linkage and commercialization in the future, even after the completion of this study.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.50 no.6
• /
• pp.212-220
• /
• 2013

Recently, there has been tremendous increase in human-computer/machine interaction system, where the goal is to provide with an appropriate service to the user at the right time with minimal human inputs for human augmented cognition system. To develop an efficient human augmented cognition system based on human computer/machine interaction, it is important to interpret the user's implicit intention, which is vague, in addition to the explicit intention. According to cognitive visual-motor theory, human eye movements and pupillary responses are rich sources of information about human intention and behavior. In this paper, we propose a novel approach for the identification of human implicit visual search intention based on eye movement pattern and pupillary analysis such as pupil size, gradient of pupil size variation, fixation length/count for the area of interest. The proposed model identifies the human's implicit intention into three types such as navigational intent generation, informational intent generation, and informational intent disappearance. Navigational intent refers to the search to find something interesting in an input scene with no specific instructions, while informational intent refers to the search to find a particular target object at a specific location in the input scene. In the present study, based on the human eye movement pattern and pupillary analysis, we used a hierarchical support vector machine which can detect the transitions between the different implicit intents - navigational intent generation to informational intent generation and informational intent disappearance.