• Archives of design research
• /
• v.18 no.2 s.60
• /
• pp.273-282
• /
• 2005

In this study, we proposed Context Cube as a conceptual model of user context, and a Multi-modal Interaction Design framework to develop ubiquitous service through understanding user context and analyzing correlation between context awareness and multi-modality, which are to help infer the meaning of context and offer services to meet user needs. And we developed a case study to verify Context Cube's validity and proposed interaction design framework to derive personalized ubiquitous service. We could understand context awareness as information properties which consists of basic activity, location of a user and devices(environment), time, and daily schedule of a user. And it enables us to construct three-dimensional conceptual model, Context Cube. Also, we developed ubiquitous interaction design process which encloses multi-modal interaction design by studying the features of user interaction presented on Context Cube.

• Journal of Multimedia Information System
• /
• v.1 no.1
• /
• pp.55-67
• /
• 2014

Image topic and emotion analysis is an important component of online image retrieval, which nowadays has become very popular in the widely growing social media community. However, due to the gaps between images and texts, there is very limited work in literature to detect one image's Topics and Emotions in a unified framework, although topics and emotions are two levels of semantics that often work together to comprehensively describe one image. In this work, a unified model, Joint Topic/Emotion Multi-Modal Hierarchical Latent Dirichlet Allocation (JTE-MMHLDA) model, which extends previous LDA, mmLDA, and JST model to capture topic and emotion information at the same time from heterogeneous data, is proposed. Specifically, a two level graphical structured model is built to realize sharing topics and emotions among the whole document collection. The experimental results on a Flickr dataset indicate that the proposed model efficiently discovers images' topics and emotions, and significantly outperform the text-only system by 4.4%, vision-only system by 18.1% in topic detection, and outperforms the text-only system by 7.1%, vision-only system by 39.7% in emotion detection.

• The Journal of Korea Robotics Society
• /
• v.8 no.3
• /
• pp.197-205
• /
• 2013

This paper presents a robotic system that provides telepresence to the visually impaired by combining real-time haptic rendering with multi-modal interaction. A virtual-proxy based haptic rendering process using a RGB-D sensor is developed and integrated into a unified framework for control and feedback for the telepresence robot. We discuss the challenging problem of presenting environmental perception to a user with visual impairments and our solution for multi-modal interaction. We also explain the experimental design and protocols, and results with human subjects with and without visual impairments. Discussion on the performance of our system and our future goals are presented toward the end.

• Journal of KIISE:Software and Applications
• /
• v.35 no.4
• /
• pp.245-254
• /
• 2008

This paper proposes a unified multi-modal input framework to interface the recognition engines such as IBM ViaVoice and Microsoft handwriting-recognition system with general window applications, particularly, for pen-input displays. As soon as user pushes a hardware button attached to the pin-input display with one hand, the current window of focus such as a internet search window and a word processor is overlaid with a transparent window covering the whole desktop; upon which user inputs handwriting with the other hand, without losing the focus of attention on working context. As well as freeform handwriting on this transparent input overlay as a sketch pad, the user can dictate some words and draw diagrams to communicate with the system.

• Smart Structures and Systems
• /
• v.6 no.5_6
• /
• pp.423-438
• /
• 2010

Wireless smart sensors enable new approaches to improve structural health monitoring (SHM) practices through the use of distributed data processing. Such an approach is scalable to the large number of sensor nodes required for high-fidelity modal analysis and damage detection. While much of the technology associated with smart sensors has been available for nearly a decade, there have been limited numbers of fulls-cale implementations due to the lack of critical hardware and software elements. This research develops a flexible wireless smart sensor framework for full-scale, autonomous SHM that integrates the necessary software and hardware while addressing key implementation requirements. The Imote2 smart sensor platform is employed, providing the computation and communication resources that support demanding sensor network applications such as SHM of civil infrastructure. A multi-metric Imote2 sensor board with onboard signal processing specifically designed for SHM applications has been designed and validated. The framework software is based on a service-oriented architecture that is modular, reusable and extensible, thus allowing engineers to more readily realize the potential of smart sensor technology. Flexible network management software combines a sleep/wake cycle for enhanced power efficiency with threshold detection for triggering network wide operations such as synchronized sensing or decentralized modal analysis. The framework developed in this research has been validated on a full-scale a cable-stayed bridge in South Korea.

• Journal of Institute of Convergence Technology
• /
• v.1 no.2
• /
• pp.1-5
• /
• 2011

It is known that as many as 15-20 in 10,000 children are diagnosed with autistic spectrum disorder. A framework of the treatment system for children with autism using affective computing technologies was proposed by Chung and Yoon. In this paper, a prototype for the framework is proposed. It consists of emotion stimulating module, multi-modal bio-signal sensing module, treatment module using virtual reality, and emotion recognition module. Primitive experiments on emotion recognition show the usefulness of the proposed system.

• Structural Engineering and Mechanics
• /
• v.64 no.6
• /
• pp.803-817
• /
• 2017

Structural health monitoring has increasingly been a focus within the civil engineering research community over the last few decades. With increasing application of sensor networks in large structures and infrastructure systems, effective use and development of robust algorithms to analyze large volumes of data and to extract the desired features has become a challenging problem. In this paper, we grasp some precautions and key points of the signal processing approach, wavelet, establish a relative reliable framework, and analyze three problems that require attention when applying wavelet based damage detection approach. The cases studies how to use optimal scales for extracting mode shapes and modal curvatures in a reinforced concrete beam and how to effectively identify damages using maximum curves of wavelet coefficient differences. Moreover, how to make a recognition based on the wavelet multi-resolution analysis, wavelet packet energy, and fuzzy sets is a meaningful topic that has been addressed in this work. The relative systematic work that compasses algorithms, structures and evaluation paves a way to a framework regarding effective structural health monitoring, orientation, decision and action.

• Smart Structures and Systems
• /
• v.16 no.1
• /
• pp.47-65
• /
• 2015

A novel finite element (FE) model updating method based on multi-resolution analysis (MRA) is proposed. The true stiffness of the FE model is considered as the superposition of two pieces of stiffness information of different resolutions: the pre-defined stiffness information and updating stiffness information. While the resolution of former is solely decided by the meshing density of the FE model, the resolution of latter is decided by the limited information obtained from the experiment. The latter resolution is considerably lower than the former. Second generation wavelet is adopted to describe the updating stiffness information in the framework of MRA. This updating stiffness in MRA is realized at low level of resolution, therefore, needs less number of updating parameters. The efficiency of the optimization process is thus enhanced. The proposed method is suitable for the identification of multiple irregular cracks and performs well in capturing the global features of the structural damage. After the global features are identified, a refinement process proposed in the paper can be carried out to improve the performance of the MRA of the updating information. The effectiveness of the method is verified by numerical simulations of a box girder and the experiment of a three-span continues pre-stressed concrete bridge. It is shown that the proposed method corresponds well to the global features of the structural damage and is stable against the perturbation of modal parameters and small variations of the damage.

• Journal of Internet Computing and Services
• /
• v.15 no.3
• /
• pp.11-19
• /
• 2014

The natural user interface/experience (NUI/NUX) is used for the natural motion interface without using device or tool such as mice, keyboards, pens and markers. Up to now, typical motion recognition methods used markers to receive coordinate input values of each marker as relative data and to store each coordinate value into the database. But, to recognize accurate motion, more markers are needed and much time is taken in attaching makers and processing the data. Also, as NUI/NUX framework being developed except for the most important intuition, problems for use arise and are forced for users to learn many NUI/NUX framework usages. To compensate for this problem in this paper, we didn't use markers and implemented for anyone to handle it. Also, we designed multi-modal NUI/NUX framework controlling voice, body motion, and facial expression simultaneously, and proposed a new algorithm of mouse operation by recognizing intuitive hand gesture and mapping it on the monitor. We implement it for user to handle the "hand mouse" operation easily and intuitively.

• Structural Engineering and Mechanics
• /
• v.59 no.4
• /
• pp.683-701
• /
• 2016

Safety is the prime concern for a high-speed railway bridge, especially when it is subjected to a collision. In this paper, an analysis framework for the dynamic responses of train-bridge systems under collision load is established. A multi-body dynamics model is employed to represent the moving vehicle, the modal decomposition method is adopted to describe the bridge structure, and the time history of a collision load is used as the external load on the train-bridge system. A (180+216+180) m continuous steel trussed-arch bridge is considered as an illustrative case study. With the vessel collision acting on the pier, the displacements and accelerations at the pier-top and the mid-span of the bridge are calculated when a CRH2 high-speed train running through the bridge, and the influence of bridge vibration on the running safety indices of the train, including derailment factors, offload factors and lateral wheel/rail forces, are analyzed. The results demonstrate that under the vessel collision load, the dynamic responses of the bridge are greatly enlarged, threatening the running safety of high-speed train on the bridge, which is affected by both the collision intensity and the train speed.