• Journal of Broadcast Engineering
• /
• v.27 no.1
• /
• pp.124-132
• /
• 2022

Recently, due to COVID-19, online remote classes and non-face-to-face exams have made it difficult to manage class attitudes and exam cheating. Therefore, there is a need for a system that automatically recognizes and detects the behavior of students online. Action recognition, which recognizes human action, is one of the most studied technologies in computer vision. In order to develop such a technology, data including human arm movement information and information about surrounding objects, which can be key information in online classes and exams, are needed. It is difficult to apply the existing dataset to this system because it is classified into various fields or consists of daily life action. In this paper, we propose a dataset that can classify attitudes in real-time online tests and classes. In addition, it shows whether the proposed dataset is correctly constructed through comparison with the existing action recognition dataset.

• Proceedings of the Korean Information Science Society Conference
• /
• 2007.10c
• /
• pp.263-266
• /
• 2007

We introduce a vision-based hand gesture recognition fer understanding musical time and patterns without extra special devices. We suggest a simple and reliable vision-based hand gesture recognition having two features First, the motion-direction code is proposed, which is a quantized code for motion directions. Second, the conducting feature point (CFP) where the point of sudden motion changes is also proposed. The proposed hand gesture recognition system extracts the human hand region by segmenting the depth information generated by stereo matching of image sequences. And then, it follows the motion of the center of the gravity(COG) of the extracted hand region and generates the gesture features such as CFP and the direction-code finally, we obtain the current timing pattern of beat and tempo of the playing music. The experimental results on the test data set show that the musical time pattern and tempo recognition rate is over 86.42% for the motion histogram matching, and 79.75% fer the CFP tracking only.

• Journal of Internet Computing and Services
• /
• v.16 no.3
• /
• pp.21-32
• /
• 2015

Since spatio-temporal local features for video representation have become an important issue of modeless bottom-up approaches in action recognition, various methods for feature extraction and description have been proposed in many papers. In particular, BoF(bag of features) has been promised coherent recognition results. The most important part for BoF is how to represent dynamic information of actions in videos. Most of existing BoF methods consider the video as a spatio-temporal volume and describe neighboring 3D interest points as complex volumetric patches. To simplify these complex 3D methods, this paper proposes a novel method that builds BoF representation as a way to learn 2D interest points directly from video data. The basic idea of proposed method is to gather feature points not only from 2D xy spatial planes of traditional frames, but from the 2D time axis called spatio-temporal frame as well. Such spatial-temporal features are able to capture dynamic information from the action videos and are well-suited to recognize human actions without need of 3D extensions for the feature descriptors. The spatio-temporal BoF approach using SIFT and SURF feature descriptors obtains good recognition rates on a well-known actions recognition dataset. Compared with more sophisticated scheme of 3D based HoG/HoF descriptors, proposed method is easier to compute and simpler to understand.

• Journal of Korea Multimedia Society
• /
• v.22 no.12
• /
• pp.1457-1465
• /
• 2019

3D convolution is to stack multiple consecutive frames to form a cube, and then apply the 3D convolution kernel in the cube. In this structure, each feature map of the convolutional layer is connected to multiple adjacent sequential frames in the previous layer, thus capturing the motion information. However, due to the changes of pedestrian posture, motion and position, the convolution at the same place is inappropriate, and when the 3D convolution kernel is convoluted in the time domain, only time domain features of three consecutive frames can be extracted, which is not a good enough to get action information. This paper proposes an action recognition method based on feature fusion of 3D convolutional neural network. Based on the VGG16 network model, sending a pre-acquired optical flow image for learning, then get the time domain features, and then the feature of the time domain is extracted from the features extracted by the 3D convolutional neural network. Finally, the behavior classification is done by the SVM classifier.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.24 no.5
• /
• pp.518-528
• /
• 2014

In this paper, we propose a method to recognize the action direction of human by developing 4D space-time (4D-ST, [x,y,z,t]) features. For this, we propose 4D space-time interest points (4D-STIPs, [x,y,z,t]) which are extracted using 3D space (3D-S, [x,y,z]) volumes reconstructed from images of a finite number of different views. Since the proposed features are constructed using volumetric information, the features for arbitrary 2D space (2D-S, [x,y]) viewpoint can be generated by projecting the 3D-S volumes and 4D-STIPs on corresponding image planes in training step. We can recognize the directions of actors in the test video since our training sets, which are projections of 3D-S volumes and 4D-STIPs to various image planes, contain the direction information. The process for recognizing action direction is divided into two steps, firstly we recognize the class of actions and then recognize the action direction using direction information. For the action and direction of action recognition, with the projected 3D-S volumes and 4D-STIPs we construct motion history images (MHIs) and non-motion history images (NMHIs) which encode the moving and non-moving parts of an action respectively. For the action recognition, features are trained by support vector data description (SVDD) according to the action class and recognized by support vector domain density description (SVDDD). For the action direction recognition after recognizing actions, each actions are trained using SVDD according to the direction class and then recognized by SVDDD. In experiments, we train the models using 3D-S volumes from INRIA Xmas Motion Acquisition Sequences (IXMAS) dataset and recognize action direction by constructing a new SNU dataset made for evaluating the action direction recognition.

• Journal of Korea Multimedia Society
• /
• v.12 no.4
• /
• pp.521-529
• /
• 2009

In this paper, we present a vision-based human primitive motion recognition method. It models the reference motion patterns, recognizes a user's motion, and measures the similarity between the reference action and the user's one. In order to recognize a motion, we provide a pattern modeling method based on the Hidden Markov Models. In addition, we provide a similarity measurement method between the reference motion and the user's one using the editing distance algorithm. Experimental results show that the recognition rate of ours is above 93%. Our method can be used in the motion recognizable games, the motion recognizable postures, and the rehabilitation training systems.

• Journal of Institute of Control, Robotics and Systems
• /
• v.16 no.10
• /
• pp.957-962
• /
• 2010

If we want to recognize the human's emotion via the facial image, first of all, we need to extract the emotional features from the facial image by using a feature extraction algorithm. And we need to classify the emotional status by using pattern classification method. The AAM (Active Appearance Model) is a well-known method that can represent a non-rigid object, such as face, facial expression. The Bayesian Network is a probability based classifier that can represent the probabilistic relationships between a set of facial features. In this paper, our approach to facial feature extraction lies in the proposed feature extraction method based on combining AAM with FACS (Facial Action Coding System) for automatically modeling and extracting the facial emotional features. To recognize the facial emotion, we use the DBNs (Dynamic Bayesian Networks) for modeling and understanding the temporal phases of facial expressions in image sequences. The result of emotion recognition can be used to rehabilitate based on biofeedback for emotional disabled.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.19 no.4
• /
• pp.562-567
• /
• 2009

As a key mechanism of the human emotion interaction, Facial Expression is a powerful tools in HRI(Human Robot Interface) such as Human Computer Interface. By using a facial expression, we can bring out various reaction correspond to emotional state of user in HCI(Human Computer Interaction). Also it can infer that suitable services to supply user from service agents such as intelligent robot. In this article, We addresses the issue of expressive face modeling using an advanced active appearance model for facial emotion recognition. We consider the six universal emotional categories that are defined by Ekman. In human face, emotions are most widely represented with eyes and mouth expression. If we want to recognize the human's emotion from this facial image, we need to extract feature points such as Action Unit(AU) of Ekman. Active Appearance Model (AAM) is one of the commonly used methods for facial feature extraction and it can be applied to construct AU. Regarding the traditional AAM depends on the setting of the initial parameters of the model and this paper introduces a facial emotion recognizing method based on which is combined Advanced AAM with Bayesian Network. Firstly, we obtain the reconstructive parameters of the new gray-scale image by sample-based learning and use them to reconstruct the shape and texture of the new image and calculate the initial parameters of the AAM by the reconstructed facial model. Then reduce the distance error between the model and the target contour by adjusting the parameters of the model. Finally get the model which is matched with the facial feature outline after several iterations and use them to recognize the facial emotion by using Bayesian Network.

• Journal of Pharmacopuncture
• /
• v.14 no.2
• /
• pp.81-91
• /
• 2011

Objectives & Methods: This suggestion was attempted to be elevated the recognition of common characteristics in disease. So, we performed to analyze the correlation of common cause of disease, characteristics of human body, and medical treatment. And the results are as follows. Results: 1. The cause of disease is consist of genetic factor, aging, habit, food of not good in health, weather, environment, deficit of the physical activity, stress and so on. 2. Generally, human has common and individual weakness. Individual weakness is appeared similar to the occurrence of volcano and lapse. 3. The correlation of disease and medical treatments is possible to explain using the quotation of the law of motion made by Isaac Newton, the great physicist. 4. When the process of the medical treatment was not progressed, the prognosis is determined by the correlation of the homeostasis(H') in human body and the homeostasis(H) of disease. 5. The prognosis of disease is determined by the relationship between the energy of disease(F) and medical treatment(F'). 6. The exact diagnosis is possible to predict the treatment sequence, and the facts that homeostasis in human body and disease, relationship between the energy of disease(F) and medical treatment(F'), action and reaction are important to determine the prognosis. 7. The careful observation of improving response and worsening action of disease becomes available for exact prognosis. Conclusion: The above described contents may be useful in clinical studies, and the concrete clinical reports about this will be made afterward.

• The Transactions of the Korean Institute of Electrical Engineers D
• /
• v.54 no.3
• /
• pp.156-165
• /
• 2005

Human being recognizes the physical world by integrating a great variety of sensory inputs, the information acquired by their own action, and their knowledge of the world using hierarchically parallel-distributed mechanism. In this paper, authors propose the sensor fusion system that can recognize multiple 3D objects from 2D projection images and tactile informations. The proposed system focuses on improving recognition performance of 3D objects. Unlike the conventional object recognition system that uses image sensor alone, the proposed method uses tactual sensors in addition to visual sensor. Neural network is used to fuse the two sensory signals. Tactual signals are obtained from the reaction force of the pressure sensors at the fingertips when unknown objects are grasped by four-fingered robot hand. The experiment evaluates the recognition rate and the number of learning iterations of various objects. The merits of the proposed systems are not only the high performance of the learning ability but also the reliability of the system with tactual information for recognizing various objects even though the visual sensory signals get defects. The experimental results show that the proposed system can improve recognition rate and reduce teeming time. These results verify the effectiveness of the proposed sensor fusion system as recognition scheme for 3D objects.