• Journal of Intelligence and Information Systems
• /
• v.28 no.1
• /
• pp.89-106
• /
• 2022

Recently, the amount of video data collected from smartphones, CCTVs, black boxes, and high-definition cameras has increased rapidly. According to the increasing video data, the requirements for analysis and utilization are increasing. Due to the lack of skilled manpower to analyze videos in many industries, machine learning and artificial intelligence are actively used to assist manpower. In this situation, the demand for various computer vision technologies such as object detection and tracking, action detection, emotion detection, and Re-ID also increased rapidly. However, the object detection and tracking technology has many difficulties that degrade performance, such as re-appearance after the object's departure from the video recording location, and occlusion. Accordingly, action and emotion detection models based on object detection and tracking models also have difficulties in extracting data for each object. In addition, deep learning architectures consist of various models suffer from performance degradation due to bottlenects and lack of optimization. In this study, we propose an video analysis system consists of YOLOv5 based DeepSORT object tracking model, SlowFast based action recognition model, Torchreid based Re-ID model, and AWS Rekognition which is emotion recognition service. Proposed model uses single-linkage hierarchical clustering based Re-ID and some processing method which maximize hardware throughput. It has higher accuracy than the performance of the re-identification model using simple metrics, near real-time processing performance, and prevents tracking failure due to object departure and re-emergence, occlusion, etc. By continuously linking the action and facial emotion detection results of each object to the same object, it is possible to efficiently analyze videos. The re-identification model extracts a feature vector from the bounding box of object image detected by the object tracking model for each frame, and applies the single-linkage hierarchical clustering from the past frame using the extracted feature vectors to identify the same object that failed to track. Through the above process, it is possible to re-track the same object that has failed to tracking in the case of re-appearance or occlusion after leaving the video location. As a result, action and facial emotion detection results of the newly recognized object due to the tracking fails can be linked to those of the object that appeared in the past. On the other hand, as a way to improve processing performance, we introduce Bounding Box Queue by Object and Feature Queue method that can reduce RAM memory requirements while maximizing GPU memory throughput. Also we introduce the IoF(Intersection over Face) algorithm that allows facial emotion recognized through AWS Rekognition to be linked with object tracking information. The academic significance of this study is that the two-stage re-identification model can have real-time performance even in a high-cost environment that performs action and facial emotion detection according to processing techniques without reducing the accuracy by using simple metrics to achieve real-time performance. The practical implication of this study is that in various industrial fields that require action and facial emotion detection but have many difficulties due to the fails in object tracking can analyze videos effectively through proposed model. Proposed model which has high accuracy of retrace and processing performance can be used in various fields such as intelligent monitoring, observation services and behavioral or psychological analysis services where the integration of tracking information and extracted metadata creates greate industrial and business value. In the future, in order to measure the object tracking performance more precisely, there is a need to conduct an experiment using the MOT Challenge dataset, which is data used by many international conferences. We will investigate the problem that the IoF algorithm cannot solve to develop an additional complementary algorithm. In addition, we plan to conduct additional research to apply this model to various fields' dataset related to intelligent video analysis.

• Proceedings of the Korean Institute of Navigation and Port Research Conference
• /
• 2003.05a
• /
• pp.181-186
• /
• 2003

The satellite navigation system is widely used for identifying a user's position regardless of weather or geographic conditions and also make effect on new technology of marine LBS(Location Based Service), which has the technology of geographic information such as the ENC. Generally, there are conceivable systems of marine LBS such as ECDIS, or ECS that use the ENC itself with powerful processor in installed type on ships bridge. Since the ENC is relatively heavy structure with dummy format for data transfer between different systems, we should reduce the ENC to small and compact size in order to use it in mobile platform. In this paper, we assumed that the mobile system like PDA, or Webpad can be used for small capability of mobile platform. However, the ENC should be updated periodically by update profile data produced by HO. If we would reduce the ENC without a consideration of update, we could not get newly updated data furthermore. As summary, we studied considerations for ENC reduction with update capability. It will make the ENC be useful in many mobile platforms for various applications.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.43 no.6 s.312
• /
• pp.26-33
• /
• 2006

Film restoration is to detect the location and extent of defected regions from a given movie film, and if present, to reconstruct the lost information of each region. It has gained increasing attention by many researchers, to support multimedia service of high quality. In general, an old film is degraded by dust, scratch, flick, and so on. Among these, the most frequent degradation is the scratch. So far techniques for the scratch restoration have been developed, but they have limited applicability when dealing with all kinds of scratches. To fully support the automatic scratch restoration, the system should be developed that can detect all kinds of scratches from a given frame of old films. This paper presents a neurual network (NN)-based texture classifier that automatically detect all kinds of scratches from frames in old films. To facilitate the detection of various scratch sizes, we use a pyramid of images generated from original frames by having the resolution at three levels. The image at each level is scanned by the NN-based classifier, which divides the input image into scratch regions and non-scratch regions. Then, to reduce the computational cost, the NN-based classifier is only applied to the edge pixels. To assess the validity of the proposed method, the experiments have been performed on old films and animations with all kinds of scratches, then the results show the effectiveness of the proposed method.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.19 no.1
• /
• pp.622-632
• /
• 2018

With the recent changes in the social environment, the growth in the so-called 'five major crimes' has been highlighted as one of the causes of anxiety in Koreans' lives. Many attempts have been made to solve this problem; however, it is still difficult to secure the location information of the socially vulnerable in emergency situations and to precisely identify the features and clothing of criminals and track them using current image analysis technology. Therefore, the development of precision positioning technology and support services along with intelligent security service technology based on spatial information has been given a high priority. This study suggested measures that could be continuously applied to link technologies and services with high linkability by analyzing technologies based on spatial information and other fields. To establish measures for linkage between intelligent security technology and other technologies and services, this study analyzed the existing technologies and research trends in intelligent security technology, and reviewed linkable services according to five criteria established to evaluate their linkability. Based on this analysis, three technologies with high linkability were ultimately selected, and measures for linkage were established. It is expected that the linkage measures derived using the objective evaluation criteria will serve as a stepping stone for promoting active technology linkage and commercialization in the future, even after the completion of this study.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.50 no.6
• /
• pp.212-220
• /
• 2013

Recently, there has been tremendous increase in human-computer/machine interaction system, where the goal is to provide with an appropriate service to the user at the right time with minimal human inputs for human augmented cognition system. To develop an efficient human augmented cognition system based on human computer/machine interaction, it is important to interpret the user's implicit intention, which is vague, in addition to the explicit intention. According to cognitive visual-motor theory, human eye movements and pupillary responses are rich sources of information about human intention and behavior. In this paper, we propose a novel approach for the identification of human implicit visual search intention based on eye movement pattern and pupillary analysis such as pupil size, gradient of pupil size variation, fixation length/count for the area of interest. The proposed model identifies the human's implicit intention into three types such as navigational intent generation, informational intent generation, and informational intent disappearance. Navigational intent refers to the search to find something interesting in an input scene with no specific instructions, while informational intent refers to the search to find a particular target object at a specific location in the input scene. In the present study, based on the human eye movement pattern and pupillary analysis, we used a hierarchical support vector machine which can detect the transitions between the different implicit intents - navigational intent generation to informational intent generation and informational intent disappearance.

• KIISE Transactions on Computing Practices
• /
• v.22 no.8
• /
• pp.375-380
• /
• 2016

The proliferation of online social networking services (OSNSs) and smartphones has enabled people to easily make friends with a large number of users in the online communities, and interact with each other. This leads to an increase in the usage rate of OSNSs. However, individuals who have immersed into their digital lives, prioritizing the virtual world against the real one, become more and more isolated in the physical world. Thus, their socialization processes that are undertaken only through lots of face-to-face interactions and trial-and-errors are apt to be neglected via 'Add Friend' kind of functions in OSNSs. In this paper, we present a friend recommendation system based on the on/off-line contextual information for the OSNS users to have more serendipitous offline interactions. In order to accomplish this, we modeled both offline information (i.e., place visit history) collected from a user's smartphone on a 3D tensor, and online social data (i.e., friend relationships) from Facebook on a matrix. We then recommended like-minded people and encouraged their offline interactions. We evaluated the users' satisfaction based on a real-world dataset collected from 43 users (12 on-campus users and 31 users randomly selected from Facebook friends of on-campus users).

• Korean Institute of Interior Design Journal
• /
• v.14 no.4 s.51
• /
• pp.87-94
• /
• 2005

Lately due to the sudden growth of the construction of an apartment complex in domestic market, many construction companies are eager to provide various public facilities to increase the apartment distribution rate. An introduction of a fitness center is popularized as a part of such development. There has been a continuing development in a private fitness based on analyzing the user's data. However a public fitness is quite different from the private in terms of a user's classification, preference rate, the pattern of use, and location which requires a different facility plan. Nevertheless, all these days a public fitness in an apartment house has been developed without a specific facility plan based on relevant materials. It is worried that such development would cause the facilities not to function as appropriate resident facilities and sink to idle ones. Therefore the purpose of this paper is to make the standard considered in facility planning in future and the base to guide an applying method. In the paper the present condition of fitness centers inside apartment houses has been compared to analyze the progress of the existing facilities and the appropriateness of the organization and the structure division. The result of the research shows that public resident facilities is not the place to perform passive role for fixed activities but one to form a community through the human relationship based on the facility. Therefore a fitness center inside the apartment complex have to be designed to provide service and leisure space as one of main community facilities for residents. To obtain this purpose, when choosing facilities of an apartment complex, first of all the residents' standard of living, apartment size, residents' distinction rate of age and sex, the needs of the times should be fully considered Secondly, the size of each facility space have to be decided based on the practical data analysis in facility use such as space preference, average staying time. Also, future living culture requires the change of the function and space according to the change of social values, so continuing research and data analysis are required to related to fitness center inside an apartment complex in order to present systematic approaching method and the paper will be expected to be a little step toward it.

• Journal of the Korean BIBLIA Society for library and Information Science
• /
• v.29 no.4
• /
• pp.91-111
• /
• 2018

This study was carried out on specialized library librarians to investigate the status and overall satisfaction of specialized services and to compare the degree of importance and satisfaction of specialized subject data, specialized service places, specialization programs, specialized subject area facilities and environment based on ISA technique in order to derive problems and improvement measures for the operation of specialized libraries. As results, first, it was found that a specialized library subject should be selected based on the characteristics, history, and resources of the library location as well as based on the demand survey of the local residents and users. Second, based on the appropriate program type of specialized subject and the results from users' demand surveys, specialized libraries should develop different types of experience-type, lecture-type, view-type and the likes. Third, specialized library space types can be divided into integrated type, independent-in-integration type, and independent type. Even though it is most ideal to provide independently specialized space, the space must be constructed considering the circumstances and subject of specialization of each library. Fourth, specialized libraries should have at least one dedicated specialist in the corresponding subject. Dedicated specialists should provide lending and reference services related to the specialization such as tidying, browsing, reading and operating specialized programs. And a dedicated specialist must be an expert in the specialized subject and also a librarian who is responsible of the general library operation.