• Educational Technology International
• /
• v.18 no.1
• /
• pp.49-71
• /
• 2017

The purpose of the study was to identify key design features of virtual reality with head-mounted displays (HMD) and touchless interface for the hearing-impaired and hard-of-hearing learners. The virtual reality based training program was aimed to help hearing-impaired learners in machine operating learning, which requires spatial understanding to operate. We developed an immersive virtual learning environment prototype with an HMD (Oculus Rift) and a touchless natural user interface (Leap Motion) to identify the key design features required to enhance virtual reality for the hearing-impaired and hard-of-hearing learners. Two usability tests of the prototype were conducted, which revealed that several features in the system need revision and that the technology presents an enormous potential to help hearing-impaired learners by providing realistic and immersive learning experiences. After the usability tests of hearing-impaired students' exploring the 3D virtual space, interviews were conducted, which also established that further revision of the system is needed, which would take into account the learners' physical as well as cognitive characteristics.

• Journal of the Institute of Convergence Signal Processing
• /
• v.19 no.4
• /
• pp.180-185
• /
• 2018

We have developed an interactive signage system based on floating hologram by combining hologram technology and ICT technology, which can be competitive to small businesses that have excellent products and services. The developed interactive signage system can be used for publicity and marketing of small business owners at low cost, introducing menus with 3D hologram images, and providing various contents responding to user's hand movements. The developed system is able to detect 10 finger movements at a rate of 290 frames per second in a range of 60cm and a range of 150 degrees. We also confirmed that the virtual touch function operates normally by dividing the user's motion recognition into the hover zone and the touch zone by the physical motion experiment of the leap motion object.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2017.11a
• /
• pp.168-169
• /
• 2017

기존의 정형적인 디스플레이 형태뿐만 아니라 비정형적인 구조의 디스플레이의 관심이 증가하고 있다. 미래에는 기존의 직사각형의 디스플레이와 더불어 비정형적인 디스플레이의 기술이 발전할 전망으로 보인다. 이러한 비정형 사이니지 기술이 발전함에 따라 비정형 영상 표출에 대한 연구가 시행될 필요성이 있다. 하지만 큐브 디스플레이와 같은 비정형 디스플레이에 바로 영상을 표출하기에는 실질적으로 구현이 어려운 점이 있다. 따라서 현실적인 하드웨어 구현이 아닌 비정형 영상 시뮬레이션을 실행할 수 있는 환경을 구축함으로써 이러한 한계를 극복하고자 한다. 본 논문은 립모션(Leap Motion)과 유니티(Unity3D)를 연동하여 제스처 인식 기반의 비정형 디스플레이 표출 시뮬레이션을 시행하였다. 사용자의 명령에 따라 큐브모형이 제어가 되어 비정형 디스플레이로 인식하고 영상을 표출한다. 이는 실제 공간에서 사용할 수 있는 조립형 큐브 디스플레이에 영상을 직접 표출하기 전 시뮬레이션으로 활용 할 수 있을 것으로 기대된다.

• Korean Journal of Construction Engineering and Management
• /
• v.20 no.2
• /
• pp.86-94
• /
• 2019

The purpose of this study was to analyzes the problems in terms of transmission the information of architecture in the current apartment housing sales phase, to suggest the way of application the technology of BIM-based virtual reality. For this purpose, each element technology is analyzed through analyzing the exiting studies and international technical trend of BIM-based virtual reality and motion recognition devices. And then this study analyze the present condition of apartment housing sales phase and it conduct the expert interviews for derive the problem and analyzing the importance and priority by AHP method. And finally, this study suggest the solution in terms of information delivery in apartment housing sales phase through plan for support and deriving the function about problems. The system of BIM-based virtual reality viewer that utilize a leap motion that was presented in this study, it is believed to be able to efficiently transfer the information of architecture as building information transmission system user-centric in the future of apartment housing sales phase.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.17 no.4
• /
• pp.1216-1233
• /
• 2023

Conventional occupational therapy (OT) is conducted under the observation of an occupational therapist, and there are limitations in measuring and analyzing details such as degree of hand tremor and movement tendency, so this important information may be lost. It is therefore difficult to identify quantitative performance indicators, and the presence of observers during performance sometimes makes the subjects feel that they have to achieve good results. In this study, by using the Unity3D and artificial intelligence (AI) speaker, we propose a system that allows the subjects to steadily use it by themselves and helps the occupational therapist objectively evaluate through quantitative data. This system is based on the OT of the sensory integration approach. And the purpose of this system is to improve children's activities of daily living by providing various feedback to induce sensory integration, which allows them to develop the ability to effectively use their bodies. A dynamic OT cognitive assessment tool for children used in clinical practice was implemented in Unity3D to create an OT environment of virtual space. The Leap Motion Controller allows users to track and record hand motion data in real time. Occupational therapists can control the user's performance environment remotely by connecting Unity3D and AI speaker. The experiment with the conventional OT tool and the system we proposed was conducted. As a result, it was found that when the system was performed without an observer, users can perform spontaneously and several times feeling ease and active mind.

• Journal of the Ergonomics Society of Korea
• /
• v.36 no.5
• /
• pp.385-394
• /
• 2017

Objective: The aim of this study is to investigate how to design effective virtual reality-based training (i.e., virtual training) in maritime safety and to present methods for enhancing interface fidelity by employing immersive interaction and 3D user interface (UI) design. Background: Emerging virtual reality technologies and hardware enable to provide immersive experiences to individuals. There is also a theory that the improvement of fidelity can improve the training efficiency. Such a sense of immersion can be utilized as an element for realizing effective training in the virtual space. Method: As an immersive interaction, we implemented gesture-based interaction using leap motion and Myo armband type sensors. Hand gestures captured from both sensors are used to interact with the virtual appliance in the scenario. The proposed 3D UI design is employed to visualize appropriate information for tasks in training. Results: A usability study to evaluate the effectiveness of the proposed method has been carried out. As a result, the usability test of satisfaction, intuitiveness of UI, ease of procedure learning, and equipment understanding showed that virtual training-based exercise was superior to existing training. These improvements were also independent of the type of input devices for virtual training. Conclusion: We have shown through experiments that the proposed interaction design results are more efficient interactions than the existing training method. The improvement of interface fidelity through intuitive and immediate feedback on the input device and the training information improve user satisfaction with the system, as well as training efficiency. Application: Design methods for an effective virtual training system can be applied to other areas by which trainees are required to do sophisticated job with their hands.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.52 no.9
• /
• pp.77-85
• /
• 2015

The finger language is the part of the sign language, which is a language system that expresses vowels and consonants with hand gestures. Korean finger language has 31 gestures and each of them needs a lot of learning models for accurate recognition. If there exist mass learning models, it spends a lot of time to search. So a real-time awareness system concentrates on how to reduce search spaces. For solving these problems, this paper suggest a hierarchy HMM structure that reduces the exploration space effectively without decreasing recognition rate. The Korean finger language is divided into 3 categories according to the direction of a wrist, and a model can be searched within these categories. Pre-classification can discern a similar finger Korean language. And it makes a search space to be managed effectively. Therefore the proposed method can be applied on the real-time recognition system. Experimental results demonstrate that the proposed method can reduce the time about three times than general HMM recognition method.

• Journal of Korea Game Society
• /
• v.17 no.6
• /
• pp.113-120
• /
• 2017

With the development of virtual reality technology, in recent years, user-friendly hand gesture interface has been more studied for natural interaction with a virtual 3D object. Most earlier studies on the hand-gesture interface are using relatively simple hand gestures. In this paper, we suggest an intuitive hand gesture interface for interaction with 3D object in the virtual reality applications. For hand gesture recognition, first of all, we preprocess various hand data and classify the data through the binary decision tree. The classified data is re-sampled and converted to the chain-code, and then constructed to the hand feature data with the histograms of the chain code. Finally, the input gesture is recognized by MCSVM-based machine learning from the feature data. To test our proposed hand gesture interface we implemented a 'Virtual Block' game. Our experiments showed about 99.2% recognition ratio of 16 kinds of command gestures and more intuitive and user friendly than conventional mouse interface.

• Korean Journal of Computational Design and Engineering
• /
• v.22 no.1
• /
• pp.89-99
• /
• 2017

In order to make a virtual endoscopy system effective for exploring the interior of the 3D model of a human organ, it is necessary to generate an accurate navigation path located inside the 3D model and to obtain consistent camera position and pose estimation along the path. In this paper, we propose an approach to virtual navigation of blood vessels, which makes proper use of orthogonal contours and skeleton curves. The approach generates the orthogonal contours and the skeleton curves from the 3D mesh model and its voxel model, all of which represent the blood vessels. For a navigation zone specified by two nodes on the skeleton curves, it computes the shortest path between the two nodes, estimates the positions and poses of a virtual camera at the nodes in the navigation zone, and interpolates the positions and poses to make the camera move smoothly along the path. In addition to keyboard and mouse input, intuitive hand gestures determined by the Leap Motion SDK are used as user interface for virtual navigation of the blood vessels. The proposed approach provides easy and accurate means for the user to examine the interior of 3D blood vessels without any collisions between the camera and their surface. With a simple user study, we present illustrative examples of applying the approach to 3D mesh models of various blood vessels in order to show its quality and usefulness.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2015.10a
• /
• pp.1019-1022
• /
• 2015

사회적 약자인 청각장애인들의 수화통역 서비스가 대체로 만족스럽게 제공되고 있지 않다. 또한 청각 장애인수 대비 수화통역가의 부족과 일반인과의 즉각적인 소통이 어려워 일상생활에 불편함을 느끼는 청각장애인들이 많다. 이의 문제점을 착안하여 본 연구에서는 청각장애인들이 휴대할 수 있는 수화 통역기를 개발하고자 한다. 청각장애인들의 손동작을 인식할 수 있는 3D센서 립모션(Leap-Motion)을 통해 수화를 인식한다. 립모션 센서와 안드로이드 스마트폰 및 태블릿과 연결하여 손 데이터 값을 분석했다. 분석한 손 데이터 값과 특정 지화(자음, 모음)와 간단한 인사말의 수화동작을 취할 때, 트래킹되는 데이터 값을 비교하여 수화를 인식하고 수화와 인식된 지화의 자음과 모음을 글자로 완성시켜, 텍스트와 음성으로 출력한다. 또한 립모션이 부착된 하드케이스 안에 스마트기기 탈착이 가능한 실용성 있는 수화통역기 제품 개발에 대한 연구를 목표로 하고 있다.