• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.13 no.2
• /
• pp.751-770
• /
• 2019

Action recognition is an essential task in computer vision due to the variety of prospective applications, such as security surveillance, machine learning, and human-computer interaction. The availability of more video data than ever before and the lofty performance of deep convolutional neural networks also make it essential for action recognition in video. Unfortunately, limited crafted video features and the scarcity of benchmark datasets make it challenging to address the multi-person action recognition task in video data. In this work, we propose a deep convolutional neural network-based Effective Hybrid Learning (EHL) framework for two-person interaction classification in video data. Our approach exploits a pre-trained network model (the VGG16 from the University of Oxford Visual Geometry Group) and extends the Faster R-CNN (region-based convolutional neural network a state-of-the-art detector for image classification). We broaden a semi-supervised learning method combined with an active learning method to improve overall performance. Numerous types of two-person interactions exist in the real world, which makes this a challenging task. In our experiment, we consider a limited number of actions, such as hugging, fighting, linking arms, talking, and kidnapping in two environment such simple and complex. We show that our trained model with an active semi-supervised learning architecture gradually improves the performance. In a simple environment using an Intelligent Technology Laboratory (ITLab) dataset from Inha University, performance increased to 95.6% accuracy, and in a complex environment, performance reached 81% accuracy. Our method reduces data-labeling time, compared to supervised learning methods, for the ITLab dataset. We also conduct extensive experiment on Human Action Recognition benchmarks such as UT-Interaction dataset, HMDB51 dataset and obtain better performance than state-of-the-art approaches.

• The Journal of Korean Institute of Communications and Information Sciences
• /
• v.36 no.7C
• /
• pp.421-427
• /
• 2011

The importance of the real-time object recognition and tracking field has been growing steadily due to rapid advancement in the computer vision applications industry. As is well known, the mean-shift algorithm is widely used in robust real-time object tracking systems. Since the mentioned algorithm is easy to implement and efficient in object tracking computation, many say it is suitable to be applied to real-time object tracking systems. However, one of the major drawbacks of this algorithm is that it always converges to a local mode, failing to perform well in a cluttered environment. In this paper, an Optical Flow-based algorithm which fits for real-time recognition of multiple moving objects is proposed. Also in the tests, the newly proposed method contributed to raising the similarity of multiple moving objects, the similarity was as high as 0.96, up 13.4% over that of the mean-shift algorithm. Meanwhile, the level of pixel errors from using the new method keenly decreased by more than 50% over that from applying the mean-shift algorithm. If the data processing speed in the video surveillance systems can be reduced further, owing to improved algorithms for faster moving object recognition and tracking functions, we will be able to expect much more efficient intelligent systems in this industrial arena.

• Journal of Intelligence and Information Systems
• /
• v.25 no.4
• /
• pp.53-65
• /
• 2019

Object tracking is one of important steps to achieve video-based surveillance systems. Object tracking is considered as an essential task similar to object detection and recognition. In order to perform object tracking, various machine learning methods (e.g., least-squares, perceptron and support vector machine) can be applied for different designs of tracking systems. In general, generative methods (e.g., principal component analysis) were utilized due to its simplicity and effectiveness. However, the generative methods were only focused on modeling the target object. Due to this limitation, discriminative methods (e.g., binary classification) were adopted to distinguish the target object and the background. Among the machine learning methods for binary classification, total error rate minimization can be used as one of successful machine learning methods for binary classification. The total error rate minimization can achieve a global minimum due to a quadratic approximation to a step function while other methods (e.g., support vector machine) seek local minima using nonlinear functions (e.g., hinge loss function). Due to this quadratic approximation, the total error rate minimization could obtain appropriate properties in solving optimization problems for binary classification. However, this total error rate minimization was based on a batch mode setting. The batch mode setting can be limited to several applications under offline learning. Due to limited computing resources, offline learning could not handle large scale data sets. Compared to offline learning, online learning can update its solution without storing all training samples in learning process. Due to increment of large scale data sets, online learning becomes one of essential properties for various applications. Since object tracking needs to handle data samples in real time, online learning based total error rate minimization methods are necessary to efficiently address object tracking problems. Due to the need of the online learning, an online learning based total error rate minimization method was developed. However, an approximately reweighted technique was developed. Although the approximation technique is utilized, this online version of the total error rate minimization could achieve good performances in biometric applications. However, this method is assumed that the total error rate minimization can be asymptotically achieved when only the number of training samples is infinite. Although there is the assumption to achieve the total error rate minimization, the approximation issue can continuously accumulate learning errors according to increment of training samples. Due to this reason, the approximated online learning solution can then lead a wrong solution. The wrong solution can make significant errors when it is applied to surveillance systems. In this paper, we propose an exactly reweighted technique to recursively update the solution of the total error rate minimization in online learning manner. Compared to the approximately reweighted online total error rate minimization, an exactly reweighted online total error rate minimization is achieved. The proposed exact online learning method based on the total error rate minimization is then applied to object tracking problems. In our object tracking system, particle filtering is adopted. In particle filtering, our observation model is consisted of both generative and discriminative methods to leverage the advantages between generative and discriminative properties. In our experiments, our proposed object tracking system achieves promising performances on 8 public video sequences over competing object tracking systems. The paired t-test is also reported to evaluate its quality of the results. Our proposed online learning method can be extended under the deep learning architecture which can cover the shallow and deep networks. Moreover, online learning methods, that need the exact reweighting process, can use our proposed reweighting technique. In addition to object tracking, the proposed online learning method can be easily applied to object detection and recognition. Therefore, our proposed methods can contribute to online learning community and object tracking, detection and recognition communities.

• Journal of Korea Society of Industrial Information Systems
• /
• v.16 no.4
• /
• pp.87-98
• /
• 2011

Detecting a moving object from videos and tracking it are basic and necessary preprocessing steps in many video systems like object recognition, context aware, and intelligent visual surveillance. In this paper, we propose a method that is able to detect a moving object quickly and accurately in a condition that background and light change in a real time. Furthermore, our system detects strongly an object in a condition that the target object is covered with other objects. For effective detection, effective Eigen-space and FCM are combined and employed, and a CONDENSATION algorithm is used to trace a detected object strongly. First, training data collected from a background image are linear-transformed using Principal Component Analysis (PCA). Second, an Eigen-background is organized from selected principal components having excellent discrimination ability on an object and a background. Next, an object is detected with FCM that uses a convolution result of the Eigen-vector of previous steps and the input image. Finally, an object is tracked by using coordinates of an detected object as an input value of condensation algorithm. Images including various moving objects in a same time are collected and used as training data to realize our system that is able to be adapted to change of light and background in a fixed camera. The result of test shows that the proposed method detects an object strongly in a condition having a change of light and a background, and partial movement of an object.

• Journal of Broadcast Engineering
• /
• v.24 no.5
• /
• pp.713-725
• /
• 2019

Automatic License Plate Recognition (ALPR) is a technology required for many applications such as Intelligent Transportation Systems and Video Surveillance Systems. Most of the studies have studied were about the detection and recognition of license plates on cars, and there is very little about detecting and recognizing license plates on motorbikes. In the case of a car, the license plate is located at the front or rear center of the vehicle and is a straight or slightly sloped license plate. Also, the background of the license plate is mainly monochromatic, and license plate detection and recognition process is less complicated. However since the motorbike is parked by using a kickstand, it is inclined at various angles when parked, so the process of recognizing characters on the motorbike license plate is more complicated. In this paper, we have developed a 2-stage YOLOv2 algorithm to detect the area of a license plate after detection of a motorbike area in order to improve the recognition accuracy of license plate for motorbike data set parked at various angles. In order to increase the detection rate, the size and number of the anchor boxes were adjusted according to the characteristics of the motorbike and license plate. Image warping algorithms were applied after detecting tilted license plates. As a result of simulating the license plate character recognition process, the proposed method had the recognition rate of license plate of 80.23% compared to the recognition rate of the conventional method(YOLOv2 without image warping) of 47.74%. Therefore, the proposed method can increase the recognition of tilted motorbike license plate character by using the adjustment of anchor boxes and the image warping which fit the motorbike license plate.

• Journal of Intelligence and Information Systems
• /
• v.29 no.2
• /
• pp.171-187
• /
• 2023

When a person's face is recognized through a recording device such as a low-pixel surveillance camera, it is difficult to capture the face due to low image quality. In situations where it is difficult to recognize a person's face, problems such as not being able to identify a criminal suspect or a missing person may occur. Existing studies on face recognition used refined datasets, so the performance could not be measured in various environments. Therefore, to solve the problem of poor face recognition performance in low-quality images, this paper proposes a method to generate high-quality images by performing image quality improvement on low-quality facial images considering various environments, and then improve the performance of facial feature point detection. To confirm the practical applicability of the proposed architecture, an experiment was conducted by selecting a data set in which people appear relatively small in the entire image. In addition, by choosing a facial image dataset considering the mask-wearing situation, the possibility of expanding to real problems was explored. As a result of measuring the performance of the feature point detection model by improving the image quality of the face image, it was confirmed that the face detection after improvement was enhanced by an average of 3.47 times in the case of images without a mask and 9.92 times in the case of wearing a mask. It was confirmed that the RMSE for facial feature points decreased by an average of 8.49 times when wearing a mask and by an average of 2.02 times when not wearing a mask. Therefore, it was possible to verify the applicability of the proposed method by increasing the recognition rate for facial images captured in low quality through image quality improvement.