• International journal of advanced smart convergence
• /
• 제9권1호
• /
• pp.193-201
• /
• 2020

The main objective of this study is to investigate the impact of additional modalities on the performance of emotion recognition using speech, facial expression and physiological measurements. In order to compare different approaches, we designed a feature-based recognition system as a benchmark which carries out linear supervised classification followed by the leave-one-out cross-validation. For the classification of four emotions, it turned out that bimodal fusion in our experiment improves recognition accuracy of unimodal approach, while the performance of trimodal fusion varies strongly depending on the individual. Furthermore, we experienced extremely high disparity between single class recognition rates, while we could not observe a best performing single modality in our experiment. Based on these observations, we developed a novel fusion method, called parametric decision fusion (PDF), which lies in building emotion-specific classifiers and exploits advantage of a parametrized decision process. By using the PDF scheme we achieved 16% improvement in accuracy of subject-dependent recognition and 10% for subject-independent recognition compared to the best unimodal results.

• 정보보호학회논문지
• /
• 제29권5호
• /
• pp.1007-1017
• /
• 2019

현재 대부분의 생체인증 시스템은 단일 생체정보를 이용하여 사용자를 인증하고 있는데, 이러한 방식은 노이즈로 인한 문제, 데이터에 대한 민감성 문제, 스푸핑, 인식률의 한계 등 많은 문제점들을 가지고 있다. 이를 해결하기 위한 방법 중 하나로 다중 생체정보를 이용하는 방법이 제시되고 있다. 다중 생체인증 시스템은 각각의 생체정보에 대해서 information fusion을 수행하여 새로운 정보를 생성한 뒤, 그 정보를 활용하여 사용자를 인증하는 방식이다. Information fusion 방법들 중에서 score-level fusion 방법을 보편적으로 많이 사용한다. 하지만 정규화 작업이 필요하다는 문제점을 갖고 있고, 데이터가 같아도 정규화 방법에 따라 인식률이 달라진다는 문제점을 갖고 있다. 이에 대한 대안으로 정규화 작업이 필요 없는 rank-level fusion 방법이 제시되고 있다. 하지만 기존의 rank-level fusion 방법들은 score-level fusion 방법보다 인식률이 낮다. 이러한 문제점을 해결하기 위해 상관계수를 이용하여 score-level fusion 방법보다 인식률이 높은 rank-level fusion 방법을 제안한다. 실험은 홍채정보(CASIA V3)와 얼굴정보(FERET V1)를 이용하여 기존의 존재하는 rank-level fusion 방법들의 인식률과 본 논문에서 제안하는 fusion 방법의 인식률을 비교하였다. 또한 score-level fusion 방법들과도 인식률을 비교하였다. 그 결과로 인식률이 약 0.3%에서 3.3%까지 향상되었다.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제13권2호
• /
• pp.810-831
• /
• 2019

Recently, continuous dimensional emotion recognition from audiovisual clues has attracted increasing attention in both theory and in practice. The large amount of data involved in the recognition processing decreases the efficiency of most bimodal information fusion algorithms. A novel algorithm, namely the incomplete Cholesky decomposition based kernel cross factor analysis (ICDKCFA), is presented and employed for continuous dimensional audiovisual emotion recognition, in this paper. After the ICDKCFA feature transformation, two basic fusion strategies, namely feature-level fusion and decision-level fusion, are explored to combine the transformed visual and audio features for emotion recognition. Finally, extensive experiments are conducted to evaluate the ICDKCFA approach on the AVEC 2016 Multimodal Affect Recognition Sub-Challenge dataset. The experimental results show that the ICDKCFA method has a higher speed than the original kernel cross factor analysis with the comparable performance. Moreover, the ICDKCFA method achieves a better performance than other common information fusion methods, such as the Canonical correlation analysis, kernel canonical correlation analysis and cross-modal factor analysis based fusion methods.

• 한국통신학회논문지
• /
• 제38A권2호
• /
• pp.174-182
• /
• 2013

Most methods for fusion-based finger vein recognition were to fuse different features or matching scores from more than one trait to improve performance. To overcome the shortcomings of "the curse of dimensionality" and additional running time in feature extraction, in this paper, we propose a finger vein recognition technology based on matching score-level fusion of a single trait. To enhance the quality of finger vein image, the contrast-limited adaptive histogram equalization (CLAHE) method is utilized and it improves the local contrast of normalized image after ROI detection. Gabor features are then extracted from eight channels based on a bank of Gabor filters. Instead of using the features for the recognition directly, we analyze the contributions of Gabor feature from each channel and apply a weighted matching score-level fusion rule to get the final matching score, which will be used for the last recognition. Experimental results demonstrate the CLAHE method is effective to enhance the finger vein image quality and the proposed matching score-level fusion shows better recognition performance.

• ETRI Journal
• /
• 제44권3호
• /
• pp.462-475
• /
• 2022

Although researchers have proposed numerous techniques for speech emotion recognition, its performance remains unsatisfactory in many application scenarios. In this study, we propose a speech emotion recognition model based on a genetic algorithm (GA)-decision tree (DT) fusion of deep and acoustic features. To more comprehensively express speech emotional information, first, frame-level deep and acoustic features are extracted from a speech signal. Next, five kinds of statistic variables of these features are calculated to obtain utterance-level features. The Fisher feature selection criterion is employed to select high-performance features, removing redundant information. In the feature fusion stage, the GA is is used to adaptively search for the best feature fusion weight. Finally, using the fused feature, the proposed speech emotion recognition model based on a DT support vector machine model is realized. Experimental results on the Berlin speech emotion database and the Chinese emotion speech database indicate that the proposed model outperforms an average weight fusion method.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• 제8권2호
• /
• pp.105-110
• /
• 2008

Human being recognizes emotion fusing information of the other speech signal, expression, gesture and bio-signal. Computer needs technologies that being recognized as human do using combined information. In this paper, we recognized five emotions (normal, happiness, anger, surprise, sadness) through speech signal and facial image, and we propose to method that fusing into emotion for emotion recognition result is applying to multimodal method. Speech signal and facial image does emotion recognition using Principal Component Analysis (PCA) method. And multimodal is fusing into emotion result applying fuzzy membership function. With our experiments, our average emotion recognition rate was 63% by using speech signals, and was 53.4% by using facial images. That is, we know that speech signal offers a better emotion recognition rate than the facial image. We proposed decision fusion method using S-type membership function to heighten the emotion recognition rate. Result of emotion recognition through proposed method, average recognized rate is 70.4%. We could know that decision fusion method offers a better emotion recognition rate than the facial image or speech signal.

• 한국지능시스템학회논문지
• /
• 제11권4호
• /
• pp.302-306
• /
• 2001

얼굴인식은 이미지의 많은 변동(표정, 조명, 얼굴의 방향 등)으로 인해 한 가지 인식 방법으로는 높은 인식률을 얻기 어렵다. 이러한 어려움을 해결하기 위해, 여러 가지 정보를 융합시키는 데이터 퓨전 방법이 연구되었다. 기존의 데이터 퓨전 방법은 보조적인 생체 정보(지문, 음성 등)를 융합하여 얼굴인식기를 보조하는 방식을 취하였다. 이 논문에서는 보조적인, 생체 정보를 사용하지 않고, 기존의 얼굴인식방법을 통해 얻어지는 상호보완적인 정보를 융합하여 사용하였다. 개별적인 얼굴인식기의 정보를 융합하기 위해, 전체적으로는 Dempster-Shafer의 퓨전이론에 근거하면서, 핵심이 되는 질량함수를 새로운 방식으로 재정의학 퓨전모델을 제안하였다. 제안된 퓨전모델을 사용하여 개별적인 얼굴인식기의 정보를 융합한 결과, 보조적인 생체정보 없이, 개별적인 얼굴인식기보다 나은 인식률을 얻을 수 있었다.

• 한국멀티미디어학회논문지
• /
• 제24권3호
• /
• pp.423-430
• /
• 2021

Aiming Graph convolutional networks (GCNs) have achieved outstanding performances on skeleton-based action recognition. However, several problems remain in existing GCN-based methods, and the problem of low recognition rate caused by single input data information has not been effectively solved. In this article, we propose a Dual-stream fusion method that combines video data and skeleton data. The two networks respectively identify skeleton data and video data and fuse the probabilities of the two outputs to achieve the effect of information fusion. Experiments on two large dataset, Kinetics and NTU-RGBC+D Human Action Dataset, illustrate that our proposed method achieves state-of-the-art. Compared with the traditional method, the recognition accuracy is improved better.

• 대한전기학회:학술대회논문집
• /
• 대한전기학회 2004년도 학술대회 논문집 정보 및 제어부문
• /
• pp.107-109
• /
• 2004

In this paper, authors propose the sensor fusion system that can recognize multiple 3D objects from 2D projection images and tactile information. The proposed system focuses on improving recognition performance of 3D object. Unlike the conventional object recognition system that uses image sensor alone, the proposed method uses tactual sensors in addition to visual sensor. Neural network is used to fuse these informations. Tactual signals are obtained from the reaction force by the pressure sensors at the fingertips when unknown objects are grasped by four-fingered robot hand. The experiment evaluates the recognition rate and the number of teaming iterations of various objects. The merits of the proposed systems are not only the high performance of the learning ability but also the reliability of the system with tactual information for recognizing various objects even though visual information has a defect. The experimental results show that the proposed system can improve recognition rate and reduce learning time. These results verify the effectiveness of the proposed sensor fusion system as recognition scheme of 3D object.

• 대한전기학회논문지:시스템및제어부문D
• /
• 제54권3호
• /
• pp.156-165
• /
• 2005

Human being recognizes the physical world by integrating a great variety of sensory inputs, the information acquired by their own action, and their knowledge of the world using hierarchically parallel-distributed mechanism. In this paper, authors propose the sensor fusion system that can recognize multiple 3D objects from 2D projection images and tactile informations. The proposed system focuses on improving recognition performance of 3D objects. Unlike the conventional object recognition system that uses image sensor alone, the proposed method uses tactual sensors in addition to visual sensor. Neural network is used to fuse the two sensory signals. Tactual signals are obtained from the reaction force of the pressure sensors at the fingertips when unknown objects are grasped by four-fingered robot hand. The experiment evaluates the recognition rate and the number of learning iterations of various objects. The merits of the proposed systems are not only the high performance of the learning ability but also the reliability of the system with tactual information for recognizing various objects even though the visual sensory signals get defects. The experimental results show that the proposed system can improve recognition rate and reduce teeming time. These results verify the effectiveness of the proposed sensor fusion system as recognition scheme for 3D objects.