• IEIE Transactions on Smart Processing and Computing
• /
• v.5 no.4
• /
• pp.267-273
• /
• 2016

Hand posture recognition is an important technique to enable a natural and familiar interface in the human-computer interaction (HCI) field. This paper introduces a hand posture recognition method using a depth camera. Moreover, the hand posture recognition method is incorporated with the Moving Picture Experts Group Rich Media User Interface (MPEG-U) Advanced User Interaction (AUI) Interface (MPEG-U part 2), which can provide a natural interface on a variety of devices. The proposed method initially detects positions and lengths of all fingers opened, and then recognizes the hand posture from the pose of one or two hands, as well as the number of fingers folded when a user presents a gesture representing a pattern in the AUI data format specified in MPEG-U part 2. The AUI interface represents a user's hand posture in the compliant MPEG-U schema structure. Experimental results demonstrate the performance of the hand posture recognition system and verified that the AUI interface is compatible with the MPEG-U standard.

• Korean Journal of Computational Design and Engineering
• /
• v.16 no.4
• /
• pp.277-284
• /
• 2011

In tangible augmented reality (AR) environments for virtual prototyping, the user interacts with virtual products by manipulating tangible objects with his or her hands, but the user often encounter awkward situations in which his or her hands are occluded by augmented virtual objects, which reduces both immersion and ease of interaction. In this paper, we present how to resolve such hand region occlusion in order to enhance natural interaction and immersive visualization. In the AR environment considered, we use two types (product-type and pointer-type) of tangible objects for tangible user interaction with a virtual product of interest. Holding the tangible objects with his or her hands, the user can create input events by touching specified regions of the product-type tangible object with the pointer-type tangible object. We developed a method for resolving hand region occlusion frequently arising during such user interaction, It first detect hand region in a real image and refines the rendered image of the virtual object by subtracting the hand region from the rendered image, Then, it superimposes the refined image onto the real image to obtain an image in which the occlusion is resolved. Incorporated into tangible AR interaction for virtual prototyping of handheld products such as cellular phones and MP3 players, the method has been found by a preliminary user study that it is not only useful to improve natural interaction and immersive visualization of virtual products, but also helpful for making the users experience the products' shapes and functions better.

• Journal of Korea Multimedia Society
• /
• v.25 no.8
• /
• pp.1175-1187
• /
• 2022

3D interaction between humans and computers has been possible with the popularization of virtual reality, and it is important to study natural and efficient virtual reality user interfaces. In user interface development, it is essential to analyze and evaluate user factors. In order to analyze the influence of factors on users who use the virtual reality color picker, this paper divides the user groups based on whether they major in art or design, whether they have experience in virtual reality, and whether they have prior knowledge about 3D color space. The color selection error and color selection time of all user groups were compared and analyzed. Although there were statistically significant differences according to the user groups, all user groups used the virtual reality color picker accurately and effectively without any difficulties.

• Journal of Korea Multimedia Society
• /
• v.15 no.4
• /
• pp.501-507
• /
• 2012

Human body motion is a non-verbal part for interaction or movement that can be used to involves real world and virtual world. In this paper, we explain a study on natural user interface (NUI) in human hand motion recognition using RGB color information and depth information by Kinect camera from Microsoft Corporation. To achieve the goal, hand tracking and gesture recognition have no major dependencies of the work environment, lighting or users' skin color, libraries of particular use for natural interaction and Kinect device, which serves to provide RGB images of the environment and the depth map of the scene were used. An improved Camshift tracking algorithm is used to tracking hand motion, the experimental results show out it has better performance than Camshift algorithm, and it has higher stability and accuracy as well.

• Korean Journal of Computational Design and Engineering
• /
• v.21 no.1
• /
• pp.20-30
• /
• 2016

Recently immersive virtual reality (VR) becomes popular due to the advanced development of I/O interfaces and related SWs for effectively constructing VR environments. In particular, natural and intuitive manipulation of 3D virtual objects is still considered as one of the most important user interaction issues. This paper presents a comparative study on the manipulation and interaction of 3D virtual objects using different interfaces and interactions in three VR environments. The comparative study includes both quantitative and qualitative aspects. Three different experimental setups are 1) typical desktop-based VR using mouse and keyboard, 2) hand gesture-supported desktop VR using a Leap Motion sensor, and 3) immersive VR by wearing an HMD with hand gesture interaction using a Leap Motion sensor. In the desktop VR with hand gestures, the Leap Motion sensor is put on the desk. On the other hand, in the immersive VR, the sensor is mounted on the HMD so that the user can manipulate virtual objects in the front of the HMD. For the quantitative analysis, a task completion time and success rate were measured. Experimental tasks require complex 3D transformation such as simultaneous 3D translation and 3D rotation. For the qualitative analysis, various factors relating to user experience such as ease of use, natural interaction, and stressfulness were evaluated. The qualitative and quantitative analyses show that the immersive VR with the natural hand gesture provides more intuitive and natural interactions, supports fast and effective performance on task completion, but causes stressful condition.

• International Journal of Contents
• /
• v.14 no.2
• /
• pp.41-44
• /
• 2018

In this paper, we propose a novel method for developing expressive typography authoring tools with personal emotions. Our goal is to implement an interactive typography system that does not rely on any particular language and provides an easy, natural user interface and allows for immediate interaction. For this purpose, we converted the text data entered by a user to image data. The image data was then used for interaction with the user. The data was synchronized with the user's skeleton information obtained from the depth camera. We decomposed the characters using the formality of language to provide a typographical movement that responds more dynamically to the user's motion. Thus, this system provides interaction as a unit of characters rather than as a whole character, allowing the user to have emotional and aesthetic emotional immersion into his or her creation.

• Journal of Internet Computing and Services
• /
• v.23 no.1
• /
• pp.19-29
• /
• 2022

Natural user interface (NUI) is used for the natural motion interface without using a specific device or tool like a mouse, keyboards, and pens. Recently, as non-contact sensor-based interaction technologies for recognizing human motion, gestures, voice, and gaze have been actively studied, an environment has been prepared that can provide more diverse contents based on various interaction methods compared to existing methods. However, as the number of sensors device is rapidly increasing, the system using a lot of sensors can suffer from a lack of computational resources. To address this problem, we proposed a real-time multi-device control system for natural interactive platform. In the proposed system, we classified two types of devices as the HC devices such as high-end commercial sensor and the LC devices such astraditional monitoring sensor with low-cost. we adopt each device manager to control efficiently. we demonstrate a proposed system works properly with user behavior such as gestures, motions, gazes, and voices.

• Korean Journal of Computational Design and Engineering
• /
• v.21 no.2
• /
• pp.159-169
• /
• 2016

In order to realize the virtual prototyping (VP) of digital products, it is important to provide the people involved in product development with the appropriate visualization and interaction of the products, and the vivid simulation of user interface (UI) behaviors in an interactive 3D virtual environment. In this paper, we propose an approach to web-based 3D virtual experience using Unity and Leap Motion. We adopt Unity as an implementation platform which easily and rapidly implements the visualization of the products and the design and simulation of their UI behaviors, and allows remote users to get an easy access to the virtual environment. Additionally, we combine Leap Motion with Unity to embody natural and immersive interaction using the user's hand gesture. Based on the proposed approach, we have developed a testbed system for web-based 3D virtual experience and applied it for the design evaluation of various digital products. Button selection test was done to investigate the quality of the interaction using Leap Motion, and a preliminary user study was also performed to show the usefulness of the proposed approach.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.17 no.9
• /
• pp.2334-2347
• /
• 2023

Metaverse services generate text data, data of ubiquitous computing, in real-time to analyze user emotions. Analysis of user emotions is an important task in metaverse services. This study aims to classify user sentiments using deep learning and pre-trained language models based on the transformer structure. Previous studies collected data from a single platform, whereas the current study incorporated the review data as "Metaverse" keyword from the YouTube and Google Play Store platforms for general utilization. As a result, the Bidirectional Encoder Representations from Transformers (BERT) and Robustly optimized BERT approach (RoBERTa) models using the soft voting mechanism achieved a highest accuracy of 88.57%. In addition, the area under the curve (AUC) score of the ensemble model comprising RoBERTa, BERT, and A Lite BERT (ALBERT) was 0.9458. The results demonstrate that the ensemble combined with the RoBERTa model exhibits good performance. Therefore, the RoBERTa model can be applied on platforms that provide metaverse services. The findings contribute to the advancement of natural language processing techniques in metaverse services, which are increasingly important in digital platforms and virtual environments. Overall, this study provides empirical evidence that sentiment analysis using deep learning and pre-trained language models is a promising approach to improving user experiences in metaverse services.

• Journal of the Ergonomics Society of Korea
• /
• v.36 no.4
• /
• pp.267-280
• /
• 2017

Objective: The aim of this study is to investigate key user experience factors of interactions for Head Mounted Display (HMD) devices in the Virtual Reality Environment (VRE). Background: Virtual reality interaction research has been conducted steadily, while interaction methods and virtual reality devices have improved. Recently, all of the virtual reality devices are head mounted display based ones. Also, HMD-based interaction types include Remote Controller, Head Tracking, and Hand Gesture. However, there is few study on usability evaluation of virtual reality. Especially, the usability of HMD-based virtual reality was not investigated. Therefore, it is necessary to study the usability of HMD-based virtual reality. Method: HMD-based VR devices released recently have only three interaction types, 'Remote Controller', 'Head Tracking', and 'Hand Gesture'. We search 113 types of research to check the user experience factors or evaluation scales by interaction type. Finally, the key user experience factors or relevant evaluation scales are summarized considering the frequency used in the studies. Results: There are various key user experience factors by each interaction type. First, Remote controller's key user experience factors are 'Ease of learning', 'Ease of use', 'Satisfaction', 'Effectiveness', and 'Efficiency'. Also, Head tracking's key user experience factors are 'Sickness', 'Immersion', 'Intuitiveness', 'Stress', 'Fatigue', and 'Ease of learning'. Finally, Hand gesture's key user experience factors are 'Ease of learning', 'Ease of use', 'Feedback', 'Consistent', 'Simple', 'Natural', 'Efficiency', 'Responsiveness', 'Usefulness', 'Intuitiveness', and 'Adaptability'. Conclusion: We identified key user experience factors for each interaction type through literature review. However, we did not consider objective measures because each study adopted different performance factors. Application: The results of this study can be used when evaluating HMD-based interactions in virtual reality in terms of usability.