• Journal of Internet Computing and Services
• /
• v.13 no.5
• /
• pp.9-19
• /
• 2012

Virtual human used as HCI in digital contents expresses his various emotions across modalities like facial expression and body posture. However, few studies considered combinations of such nonverbal multimodal in emotion perception. Computational engine models have to consider how a combination of nonverbal modal like facial expression and body posture will be perceived by users to implement emotional virtual human, This paper proposes the impacts of nonverbal multimodal in design of emotion expressed virtual human. First, the relative impacts are analysed between different modals by exploring emotion recognition of modalities for virtual human. Then, experiment evaluates the contribution of the facial and postural congruent expressions to recognize basic emotion categories, as well as the valence and activation dimensions. Measurements are carried out to the impact of incongruent expressions of multimodal on the recognition of superposed emotions which are known to be frequent in everyday life. Experimental results show that the congruence of facial and postural expression of virtual human facilitates perception of emotion categories and categorical recognition is influenced by the facial expression modality, furthermore, postural modality are preferred to establish a judgement about level of activation dimension. These results will be used to implementation of animation engine system and behavior syncronization for emotion expressed virtual human.

• 한국HCI학회:학술대회논문집
• /
• 2006.02a
• /
• pp.1011-1018
• /
• 2006

멀티모달 인터페이스는 인간의 제스처, 시선, 손의 움직임, 행동의 패턴, 음성, 물리적인 위치 등 인간의 자연스러운 행동들에 대한 정보를 해석하고 부호화하는 인지기반 기술이다. 본 논문에서는 제스처와 음성, 터치를 이용한 3D 오브젝트 기반의 멀티모달 인터페이스를 설계, 구현한다. 서비스 도메인은 스마트 홈이며 사용자는 3D 오브젝트 직접조작을 통해 원격으로 가정의 오브젝트들을 모니터링하고 제어할 수 있다. 멀티모달 인터랙션 입출력 과정에서는 여러 개의 모달리티를 병렬적으로 인지하고 처리해야 하기 때문에 입출력 과정에서 각 모달리티의 조합과 부호화 방법, 입출력 형식 등이 문제시된다. 본 연구에서는 모달리티들의 특징과 인간의 인지구조 분석을 바탕으로 제스처, 음성, 터치 모달리티 간의 입력조합방식을 제시하고 멀티모달을 이용한 효율적인 3D Object 인터랙션 프로토타입을 설계한다.

• Journal of the Korean Society for Railway
• /
• v.12 no.6
• /
• pp.1067-1075
• /
• 2009

The Bimodal Tram being developed in Korea is a new transit system that provides both benefits of rail and bus systems in terms of accessibility and fixed time schedule. This study accomplished a feasibility study for introducing the Bimodal Tram system by considering the travel demand and existing transit system at a city level based on its population. The feasibility of Bimodal Tram system varies depending on cities with respect to demographic scale, mode shares, and operational distances. The strategies introduced here will help any cities where transport agencies are trying to introduce the Bimodal Tram as a new transit system.

• Annual Conference on Human and Language Technology
• /
• 2023.10a
• /
• pp.258-262
• /
• 2023

과제 지향 대화 시스템은 발화 의도 및 요구사항을 파악하여 사용자가 원하는 과제를 달성한다는 점에서 유용하다. 대화 상태 추적은 과제 지향 대화 시스템의 핵심 모듈이며, 최근에는 텍스트뿐만 아니라 시각 정보까지 활용하여 대화 상태를 추적하는 멀티모달 대화 상태 추적 연구가 활발히 진행되는 중이다. 본 논문에서는 멀티모달 공황장애 상담 대화 속 내담자의 상태를 추적하는 과제를 제안하였다. ChatGPT를 통한 멀티모달 공황장애 상담 과제 지향 대화 데이터셋 구축 프레임워크와, 구축한 데이터셋의 품질을 증명하기 위한 분석도 함께 제시하였다. 사전학습 언어 모델인 GPT-2를 벤치마크 데이터셋에 대해 학습한 성능을 측정함으로써 향후 멀티모달 대화 추적 성능이 능가해야 할 베이스라인 성능을 제시하였다.

• 한국HCI학회:학술대회논문집
• /
• 2009.02a
• /
• pp.278-284
• /
• 2009

Multimodal data to have several modalities such as videos, images, sounds and texts for one contents is increasing. Since this type of data has ill-defined format, it is not easy to represent the crossmodal information for them explicitly. So, we proposed new method to extract and analyze vision-language crossmodal association information using the documentaries video data about the nature. We collected pairs of images and captions from 3 genres of documentaries such as jungle, ocean and universe, and extracted a set of visual words and that of text words from them. We found out that two modal data have semantic association on crossmodal association information from this analysis.

• Proceedings of the Korean Society for Noise and Vibration Engineering Conference
• /
• 2006.05a
• /
• pp.458-497
• /
• 2006

• Proceedings of the KSR Conference
• /
• 2009.05a
• /
• pp.1799-1804
• /
• 2009

Bimodal Tram is spotlighted for alternative public transportation system which have punctualness, efficiency, and convenience running between capacity of bus and light rail transit. It have the advantage of automatic operation system when it runs at exclusive right-of-way as like as railway, runs as same as a bus at road concurrently. However, concrete construction plan is not established since the law as well as government subsidy are not defined. For activating bimodal tram construction, this paper analyzed as below. 1) understanding obstacles of activating construction through current status of local governments' bimodal tram promotion plan, 2) finding essential elements for legislation through transportation mode characteristics comparison and system rules analysis, 3) related foreign law analysis.

• Annual Conference on Human and Language Technology
• /
• 2019.10a
• /
• pp.449-453
• /
• 2019

미디어방송이 다양해지고, 웹에서 소비되는 콘텐츠들 또한 멀티미디어 중심으로 재편되는 경향에 힘입어 인공지능 연구에 멀티미디어 콘텐츠를 적극적으로 활용하고자 하는 시도들이 시작되고 있다. 본 논문은 다양한 형태의 멀티모달 정보를 하나의 동영상 콘텐츠에 연계하여 분석하여, 통합된 형태의 융합정보 데이터셋을 구축한 연구를 소개하고자 한다. 구축한 인공지능 학습용 데이터셋은 영상/음성/언어 정보가 함께 있는 멀티모달 콘텐츠에 상황/의도/감정 정보 추론에 필요한 다양한 의미정보를 부착하여 활용도가 높은 인공지능 영상 데이터셋을 구축하여 공개하였다. 본 연구의 결과물은 한국어 대화처리 연구에 부족한 공개 데이터 문제를 해소하는데 기여하였고, 한국어를 중심으로 다양한 상황 정보가 함께 구축된 데이터셋을 통하여 다양한 상황 분석 기반 대화 서비스 응용 기술 연구에 활용될 것으로 기대할 수 있다.

• The Journal of the Korea Contents Association
• /
• v.18 no.10
• /
• pp.11-19
• /
• 2018

This paper is about multimodal storytelling experience applying Virtual Reality technology in museum. Specifically, this research argues virtual reality in both intuitive understanding of history also multimodal experience in the space. This research investigates cases regarding use of virtual reality in museum sector. As a research method, this paper conducts a literature review regarding multimodal experience and examples applying virtual reality related technologies in museum. Based on the literature review to investigate the concept necessary with its related cases. Based on the investigation, this paper suggests constructing elements for multimodal storytelling based on VR. Ultimately, this paper suggests the elements of building VR storytelling where dynamic audio-visual and interaction mode combines with historical resources for diverse audiences.

• Journal of Korean Society of Transportation
• /
• v.30 no.3
• /
• pp.17-30
• /
• 2012

This study analyzes the basic characteristics needed for motivating modal shift from truck to rail. The object of the study is to consider the resistance factor for container transported inland and the object point is the transfer node where modal shift occurs. The main contents of the study are as follows. First, the study looks at what shippers, engaged in inland transport of container, think of current transfer node environment. From this, the study analyzes what resistance factors are attributable to the lack of motivation in modal shift from truck to rail. Second, the study estimates mode choice models in which the resistance factors at transfer node are applied. Also by using this, the study draws out elasticities, and analyzes characteristics of resistantance factors in modal shift. Third, as its result, the study suggests methods to promote modal-shifting container transport from truck to rail.