• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제13권7호
• /
• pp.3599-3619
• /
• 2019

In human activity recognition system both static and motion information play crucial role for efficient and competitive results. Most of the existing methods are insufficient to extract video features and unable to investigate the level of contribution of both (Static and Motion) components. Our work highlights this problem and proposes Static-Motion fused features descriptor (SMFD), which intelligently leverages both static and motion features in the form of descriptor. First, static features are learned by two-stream 3D convolutional neural network. Second, trajectories are extracted by tracking key points and only those trajectories have been selected which are located in central region of the original video frame in order to to reduce irrelevant background trajectories as well computational complexity. Then, shape and motion descriptors are obtained along with key points by using SIFT flow. Next, cholesky transformation is introduced to fuse static and motion feature vectors to guarantee the equal contribution of all descriptors. Finally, Long Short-Term Memory (LSTM) network is utilized to discover long-term temporal dependencies and final prediction. To confirm the effectiveness of the proposed approach, extensive experiments have been conducted on three well-known datasets i.e. UCF101, HMDB51 and YouTube. Findings shows that the resulting recognition system is on par with state-of-the-art methods.

• 한국HCI학회:학술대회논문집
• /
• 한국HCI학회 2007년도 학술대회 1부
• /
• pp.667-674
• /
• 2007

Facial expressions provide significant clues about one's emotional state; however, it always has been a great challenge for machine to recognize facial expressions effectively and reliably. In this paper, we report a method of feature-based adaptive motion energy analysis for recognizing facial expression. Our method optimizes the information gain heuristics of ID3 tree and introduces new approaches on (1) facial feature representation, (2) facial feature extraction, and (3) facial feature classification. We use minimal reasonable facial features, suggested by the information gain heuristics of ID3 tree, to represent the geometric face model. For the feature extraction, our method proceeds as follows. Features are first detected and then carefully "selected." Feature "selection" is finding the features with high variability for differentiating features with high variability from the ones with low variability, to effectively estimate the feature's motion pattern. For each facial feature, motion analysis is performed adaptively. That is, each facial feature's motion pattern (from the neutral face to the expressed face) is estimated based on its variability. After the feature extraction is done, the facial expression is classified using the ID3 tree (which is built from the 1728 possible facial expressions) and the test images from the JAFFE database. The proposed method excels and overcomes the problems aroused by previous methods. First of all, it is simple but effective. Our method effectively and reliably estimates the expressive facial features by differentiating features with high variability from the ones with low variability. Second, it is fast by avoiding complicated or time-consuming computations. Rather, it exploits few selected expressive features' motion energy values (acquired from intensity-based threshold). Lastly, our method gives reliable recognition rates with overall recognition rate of 77%. The effectiveness of the proposed method will be demonstrated from the experimental results.

• International Journal of Contents
• /
• 제4권2호
• /
• pp.24-28
• /
• 2008

In this paper, we propose a new approach for content-based video retrieval using non-parametric based motion classification in the shot-based video indexing structure. Our system proposed in this paper has supported the real-time video retrieval using spatio-temporal feature comparison by measuring the similarity between visual features and between motion features, respectively, after extracting representative frame and non-parametric motion information from shot-based video clips segmented by scene change detection method. The extraction of non-parametric based motion features, after the normalized motion vectors are created from an MPEG-compressed stream, is effectively fulfilled by discretizing each normalized motion vector into various angle bins, and by considering the mean, variance, and direction of motion vectors in these bins. To obtain visual feature in representative frame, we use the edge-based spatial descriptor. Experimental results show that our approach is superior to conventional methods with regard to the performance for video indexing and retrieval.

• 한국컴퓨터정보학회논문지
• /
• 제3권4호
• /
• pp.27-34
• /
• 1998

본 논문에서는 움직임 벡터와 빛의 세기를 이용하여 비디오의 인덱싱과 검색 기법에 대하여 제안한다. 본 논문에서는 움직임 벡터의 특징과 빛의 세기를 계산하여 각 샷 당하나의 대표프레임을 추출하였다. 각각의 대표프레임은 빛의 흐름을 계산하였다. 즉 움직임벡터의 특징은 빛의 흐름으로부터 얻어냈고, BMA 는 움직임 벡터를 찾기 위해 사용했다. 그리고 빛의 세기 값을 히스토그램으로 변환 한 후 컷 검출에 사용하였다. 비디오 프레임의움직임 벡터와 빛의 세기 특징을 기반으로 비디오 데이터를 구성하고 인덱싱 하였다. 비디오 데이터베이스는 비디오의 접근을 위해 내용기반을 제공하고, 인덱스 특징은 B+ 트리 검색을 사용했고, 내부적으로 구성되어 단 노드 방식으로 저장되어 컴퓨터 저장장치에 직접 접근할 수 있게 했다. 본 논문에서는 비디오 데이터 모델을 기반으로 한 비디오 인덱스의 문제를 정의하였다.

• 인터넷정보학회논문지
• /
• 제8권3호
• /
• pp.119-129
• /
• 2007

본 논문에서는 두 대의 카메라를 직각으로 배치하여 얻은 동영상에서 인체의 실루엣을 이용하여 동작을 인식하는 방법을 제안한다. 제안된 시스템은 실루엣에서 전역 특징과 지역 특징을 추출하며, 이 특징들은 정적인 프레임에만 있느냐에 따라 정적 특징과 동적 특징으로 다시 나뉜다. 추출된 특징들은 RBF 신경망을 훈련시키기 위해 사용된다. 제안된 신경망은 정적 특징을 입력층으로 보내고, 동적 특징은 인식을 위한 추가적인 특징으로 이용한다. 본 논문에서 제안된 신경망 동작 인식 시스템은 유아들의 동작 교육에 적용되었다. 동작 교육을 위해 제시되는 기본 동작은 걷기, 뛰기, 앙감질 등의 이동 동작과 구부리기, 뻗기, 균형 잡기, 회전하기 등 비 이동 동작으로 구분된다. 제안된 시스템은 동작교육을 위해 7가지 기본 동작을 학습시킨 신경망으로 성공적으로 동작 인식을 하였다. 제안된 시스템은 유아의 공간감각 계발을 위한 동작교육 시스템에 활용될 수 있다.

• 로봇학회논문지
• /
• 제3권4호
• /
• pp.263-269
• /
• 2008

Visual odometry is a popular approach to estimating robot motion using a monocular or stereo camera. This paper proposes a novel visual odometry scheme using a stereo camera for robust estimation of a 6 DOF motion in the dynamic environment. The false results of feature matching and the uncertainty of depth information provided by the camera can generate the outliers which deteriorate the estimation. The outliers are removed by analyzing the magnitude histogram of the motion vector of the corresponding features and the RANSAC algorithm. The features extracted from a dynamic object such as a human also makes the motion estimation inaccurate. To eliminate the effect of a dynamic object, several candidates of dynamic objects are generated by clustering the 3D position of features and each candidate is checked based on the standard deviation of features on whether it is a real dynamic object or not. The accuracy and practicality of the proposed scheme are verified by several experiments and comparisons with both IMU and wheel-based odometry. It is shown that the proposed scheme works well when wheel slip occurs or dynamic objects exist.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제11권2호
• /
• pp.1118-1133
• /
• 2017

Computer vision-based human activity recognition (HAR) has become very famous these days due to its applications in various fields such as smart home healthcare for elderly people. A video-based activity recognition system basically has many goals such as to react based on people's behavior that allows the systems to proactively assist them with their tasks. A novel approach is proposed in this work for depth video based human activity recognition using joint-based motion features of depth body shapes and Deep Belief Network (DBN). From depth video, different body parts of human activities are segmented first by means of a trained random forest. The motion features representing the magnitude and direction of each joint in next frame are extracted. Finally, the features are applied for training a DBN to be used for recognition later. The proposed HAR approach showed superior performance over conventional approaches on private and public datasets, indicating a prominent approach for practical applications in smartly controlled environments.

• 대한전자공학회논문지SP
• /
• 제37권5호
• /
• pp.31-41
• /
• 2000

본 논문은 입력 영상에 따라 적응적으로 구해진 임계 값을 이용하여 움직임을 검출하는 블럭 단위 움직임 검출 기법을 제안한다 우선, 현재 영상을 블럭의 크기에 따라 블럭화 한 후 각 블럭의 특정 값을 구하고 이 전 영상에서 저장된 블럭 특정 값과의 차이 값을 구한 다음 임계 값을 이용하여 움직임을 검출한다. 본 논문 에서는 적응적인 임계 값을 구하기 위해서 움직임 벡터를 이용하여 움직임 블럭과 배경 블럭을 구분하고 각 각의 영역에 대한 통계척인 분포를 해석하여 움직임 판별을 위한 각 특정 값의 임계 값을 입력 영상에 따라 자동 조정한다 모의 실험을 통하여 블럭의 크기가 움직임 검출 성능에 미치는 영향, 노이즈의 영향, 특정 값의 종류에 따른 검출의 정확도 기존의 움직임 검출 알고리즘과의 성능을 비교 분석한다

• 한국정보통신학회논문지
• /
• 제8권3호
• /
• pp.694-701
• /
• 2004

본 논문에서는 능동카메라 환경에서 카메라의 움직임에 의해 유발되는 광역움직임(global motion)과 이동물체에 의해 발생하는 지역움직임(local motion)을 분리한 후, 카메라 팬틸트를 제어하여 물체를 추적하는 특징기반의 추적 시스템을 제안했다. 제안한 시스템은 블록기반 움직임 계측을 통해 연속한 2 프레임 사이의 이동 움직임을 찾고, 이 움직임에서 카메라의 움직임으로 인한 광역 움직임을 제거함으로써 전경물체의 지역 움직임만을 추적한다. 이때, 배경만의 움직임만으로 카메라 움직임을 강건하게 계측하기 위하여, 블록기반 움직임에서 배경움직임을 분류하기 위한 지배적인 움직임 추출방법을 제시한다. 또한 분리된 지역움직임으로부터 잡음물체의 움직임을 제거하기 위하여 꼭지점 특징의 추적궤적 속성에 따른 군집화 알고리즘을 제안한다. 제안한 추적시스템은 여러가지 실험에서 좋은 결과를 보였다.

• 제어로봇시스템학회논문지
• /
• 제21권6호
• /
• pp.547-552
• /
• 2015

In this paper, an ARS-EKF based motion counting algorithm for repetitive exercises such as calisthenics is proposed using a smartwatch. Raw sensor signals from accelerometers and gyroscopes are widely used for conventional smartwatch counting algorithms based on pattern recognition. However, generated features from raw data are not intuitive to reflect the movement of motions. The proposed motion counter algorithm is composed of navigation based feature generation and counting with error correction. The candidate features for each activity are velocity and attitude calculated through an ARS-EKF algorithm. In order to select those features which reveal the characteristics of each motion, an exercise frame from the initial sensor frame is introduced. Counting processes are basically based on the zero crossing method, and misdetected counts are eliminated via simple classification algorithms considering the frequency of the counted motions. Experimental results show that the proposed algorithm efficiently and accurately counts the number of exercises.