• Proceedings of the Korea Multimedia Society Conference
• /
• 2012.05a
• /
• pp.71-72
• /
• 2012

• Science of Emotion and Sensibility
• /
• v.14 no.4
• /
• pp.627-636
• /
• 2011

The purpose of this study is to develop the multi-modal interaction system. This system provides realistic and an immersive experience through multi-modal interaction. The system recognizes user behavior, intention, and attention, which overcomes the limitations of uni-modal interaction. The multi-modal interaction system is based upon gesture interaction methods, intuitive gesture interaction and attention evaluation technology. The gesture interaction methods were based on the sensors that were selected to analyze the accuracy of the 3-D gesture recognition technology using meta-analysis. The elements of intuitive gesture interaction were reflected through the results of experiments. The attention evaluation technology was developed by the physiological signal analysis. This system is divided into 3 modules; a motion cognitive system, an eye gaze detecting system, and a bio-reaction sensing system. The first module is the motion cognitive system which uses the accelerator sensor and flexible sensors to recognize hand and finger movements of the user. The second module is an eye gaze detecting system that detects pupil movements and reactions. The final module consists of a bio-reaction sensing system or attention evaluating system which tracks cardiovascular and skin temperature reactions. This study will be used for the development of realistic digital entertainment technology.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2016.04a
• /
• pp.192-194
• /
• 2016

스마트폰 사용자 수가 늘어남으로써 스마트폰으로 개인에 맞는 서비스를 제공하는 것은 중요한 연구 주제가 되었고 사용자 개인의 데이터를 이용하여 기호나 취향에 맞는 상품 및 서비스 제공에 대한 개발이 활발히 이루어지고 있다. 개인에게 적합한 서비스를 제공하기 위해 데이터를 수집하는 것, 정보를 추출하는 것 및 상황 행위에 대한 특정을 하고 사용자에 대한 로그(log)를 축적하고 분석하는 작업이 가장 중요하다. 본 논문에서는 안드로이드 환경 기반의 멀티모달 센서 및 문자/통화/사진/음악 이용 로그를 활용하여 라이프로그를 저장하고 사용자의 취향을 예측할 수 있는 시스템을 제안한다. 스마트폰의 향상된 성능, 추가된 다양한 기능에 따라 생성되는 방대한 양의 데이터들을 수집하고 상황인지, 행위인지 모듈을 통하여 사용자의 상황과 행위를 특정 짓는다. 결과 또는 키워드 들을 데이터와 함께 태깅하고 에피소드 형식으로 레코드를 체계적이고 정확하게 저장한다. 이러한 시스템을 이용해 저장된 라이프로그 및 개인맞춤형 정보화 모델은 개인 취향에 최적화된 서비스/상품 제공 연구에 활용 될 수 있도록 시스템 구현을 진행할 예정이다.

• Journal of Intelligence and Information Systems
• /
• v.29 no.1
• /
• pp.203-214
• /
• 2023

The Korea Customs Service is efficiently handling business with an electronic customs system that can effectively handle one-stop business. This is the case and a more effective method is needed. Import and export require HS Code (Harmonized System Code) for classification and tax rate application for all goods, and item classification that classifies the HS Code is a highly difficult task that requires specialized knowledge and experience and is an important part of customs clearance procedures. Therefore, this study uses various types of data information such as product name, product description, and product image in the item classification request form to learn and develop a deep learning model to reflect information well based on Multimodal representation learning. It is expected to reduce the burden of customs duties by classifying and recommending HS Codes and help with customs procedures by promptly classifying items.

• Journal of Korea Multimedia Society
• /
• v.13 no.2
• /
• pp.225-236
• /
• 2010

This study proposes a method for the recognition of emergency situation according to the bimodal information of camera image sensor and gravity sensor. This method can recognize emergency condition by mutual cooperation and compensation between sensors even when one of the sensors malfunction, the user does not carry gravity sensor, or in the place like bathroom where it is hard to acquire camera images. This paper implemented HMM(Hidden Markov Model) based learning and recognition algorithm to recognize actions such as walking, sitting on floor, sitting at sofa, lying and fainting motions. Recognition rate was enhanced when image feature vectors and gravity feature vectors are combined in learning and recognition process. Also, this method maintains high recognition rate by detecting moving object through adaptive background model even in various illumination changes.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2018.05a
• /
• pp.428-430
• /
• 2018

본 논문에서는 독감확산 예측을 위한 웨어러블 센서를 이용한 기침 감지 모델을 제안한다. 서로 상이한 기침 신체데이터를 사용하고 기침 감지 알고리즘의 구현없이 기계가 학습하는 방식인 멀티모달 DNN을 이용하여 설계하였다. 또한 웨어러블 센서를 통해 실생활의 기침 오디오 데이터와 기침 3축 가속도 데이터를 수집하였고, 두 개의 데이터중 하나의 데이터만으로도 감지를 위한 학습이 가능토록하기 위해 각각 MFCC와 FFT를 이용하여 특징 벡터를 추출하는 방법을 이용하였다.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2021.10a
• /
• pp.397-399
• /
• 2021

In this study, a basic study on the development of a multi-stimulation system was conducted to suppress lung cancer cell proliferation. Stimulation was applied to lung cancer cells using a photo-stimulating system and ultrasonic waves that generate a specific frequency, and the effect of inhibiting proliferation of cells was imaged and quantitatively evaluated. As a result of the experiment, when a single LED, single ultrasound stimulus were applied and ultrasound and LED stimuli were applied at the same time, meaningful results were shown in the proliferation rate of lung cancer cells.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.23 no.1
• /
• pp.27-32
• /
• 2019

Research that implements the classification process through Deep Learning algorithm, one of the outstanding human abilities, includes a unimodal model, a multi-modal model, and a multi-modal method using music videos. In this study, the results were better by suggesting a system to analyze each song's spectrum into short samples and vote for the results. Among Deep Learning algorithms, CNN showed superior performance in the category of music genre compared to RNN, and improved performance when CNN and RNN were applied together. The system of voting for each CNN result by Deep Learning a short sample of music showed better results than the previous model and the model with Softmax layer added to the model performed best. The need for the explosive growth of digital media and the automatic classification of music genres in numerous streaming services is increasing. Future research will need to reduce the proportion of undifferentiated songs and develop algorithms for the last category classification of undivided songs.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.14 no.4
• /
• pp.157-163
• /
• 2014

User's sensitivity is recognized as a very important parameter for communication between company, government and personnel. Especially in many studies, researchers use voice tone, voice speed, facial expression, moving direction and speed of body, and gestures to recognize the sensitivity. Multiplex modality is more precise than single modality however it has limited recognition rate and overload of data processing according to multi-sensing also an excellent algorithm is needed to deduce the sensing value. That is as each modality has different concept and property, errors might be happened to convert the human sensibility to standard values. To deal with this matter, the sensibility expression modality is needed to be extracted using technologies like analyzing of relational network, understanding of context and digital filter from multiplex modality. In specific situation to recognize the sensibility if the priority modality and other surrounding modalities are processed to implicit values, a robust system can be composed in comparison to the consuming of computer resource. As a result of this paper, it is proposed how to assign the weight of multiplex modality using implicit data.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.46 no.4
• /
• pp.76-81
• /
• 2009

We propose an effective emotion recognition algorithm based on the minimum classification error (MCE) incorporating multi-modal system The emotion recognition is performed based on a Gaussian mixture model (GMM) based on MCE method employing on log-likelihood. In particular, the reposed technique is based on the fusion of feature vectors based on voice signal and galvanic skin response (GSR) from the body sensor. The experimental results indicate that performance of the proposal approach based on MCE incorporating the multi-modal system outperforms the conventional approach.