• Proceedings of the Korean Society of Computer Information Conference
• /
• 2012.07a
• /
• pp.449-452
• /
• 2012

본 논문에서는 인간이 환경에 대한 상황을 직접적으로 파악할 수 있는 시각 정보를 제공하기 위해 다중 카메라를 이용한 사용자 시각기반 어라운드 뷰를 개발하였다. 4대의 하향식 경사 카메라를 통하여 영상을 획득하고 켈리브레이션한다. 렌즈의 왜곡을 보정하고 호모그라피 행렬을 계산하여 지표면과 수평이 되는 관점으로 영상을 변환한다. 그 결과 사용자에게 종합적 상황정보 획득이 용이하도록 정보화하기 위한 위성 영상 관점의 정보를 획득할 수 있다. 그리고 4대의 카메라를 동시에 사용하기 위한 하드웨어적 한계를 극복하고자 영상처리가 가능한 임베디드 카메라 모듈을 개발하였다. 사용자-로봇 상호작용을 위해 버튼 및 조이스틱과 같은 기계적 입력장치를 사용하지 않고 사용자의 자연스러운 제스처를 통하여 제어 명령을 입력할 수 있는 터치 패드를 사용하여 사용자 인터페이스를 구축하였다. 개발한 시스템은 시 공간적 한계를 극복하고 원격에서 로봇의 상황정보를 획득하여 사용자 친화적인 로봇제어를 할 수 있다. 위의 내용들을 검증하기 위하여 같은 상황 환경에서의 기존의 시스템과 비교 실험을 진행하였고 실험 결과를 통하여 제안한 시스템의 효용성을 검증하였다.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.18 no.3
• /
• pp.541-550
• /
• 2014

This paper describes a multi-scale user interaction based tiled display visualization system using multiple input devices for monitoring and analyzing Geostationary Ocean Color Imager (GOCI) observation satellite imagery. This system provides multi-touch screen, Kinect motion sensing, and moblie interface for multiple users to control the satellite imagery either in front of the tiled display screen or far away from a distance to view marine environmental or climate changes around Korean peninsular more effectively. Due to a large amount of memory required for loading high-resolution GOCI satellite images, we employed the multi-level image load technique where the image was divided into small tiled images in order to reduce the load on the system and to be operated smoothly by user manipulation. This system performs the abstraction of common input information from multi-user Kinect motion and gestures, multi-touch points and mobile interaction information to enable a variety of user interactions for any tiled display application. In addition, the unit of time corresponding to the selected date of the satellite images are sequentially displayed on the screen and multiple users can zoom-in/out, move the imagery and select buttons to trigger functions.

• Proceedings of the Korea Contents Association Conference
• /
• 2009.05a
• /
• pp.175-178
• /
• 2009

In this paper, we describe about interactive electronic book utilizing tabletop display interface and digi-pet. Because of this system can define a lot of gestures using touch point of finger, help that user experiences electronic book effectively more than existent input device. Also, support 1 users in 1 input device in existent mode, but this system can expect various effect because support Multi-user. The digi-pet(digital pet) is physical tools that convey emotion between tabletop display platform and user. And this shows various results acting as assistance because becomes master of story of imagination and participate directly on talk and makes change of story. In this paper, we described about oriented electronic book system the future that combines tabletop display and Digi-pet. This system is expected that the usefulness is increased by leaps and bounds along with technology development forward.

• Journal of the Ergonomics Society of Korea
• /
• v.27 no.4
• /
• pp.109-117
• /
• 2008

Compared to the existing PC which uses a mouse and a keyboard, the touchscreen-based portable PC allows the user to use fingers, requiring new operation methods. However, current touchscreen-based web browser operations in many cases involve merely having fingers move simply like a mouse and click, or not corresponding well to the user's sensitivity and the structure of one's index finger, making itself difficult to be used during walking. Therefore, the goal of this study is to develop finger gestures which facilitate the interaction between the interface and the user, and make the operation easier. First, based on the frequency of usage in the web browser and preference, top eight functions were extracted. Then, the users' structural knowledge was visualized through sketch maps, and the finger gestures which were applicable in touchscreens were derived through the Meaning in Mediated Action method. For the front/back page, and up/down scroll functions, directional gestures were derived, and for the window closure, refresh, home and print functions, letter-type and icon-type gestures were drawn. A validation experiment was performed to compare the performance between existing operation methods and the proposed one in terms of execution time, error rate, and preference, and as a result, directional gestures and letter-type gestures showed better performance than the existing methods. These results suggest that not only during the operation of touchscreen-based web browser in portable PC but also during the operation of telematics-related functions in automobile, PDA and so on, the new gestures can be used to make operation easier and faster.

• KIPS Transactions on Software and Data Engineering
• /
• v.9 no.9
• /
• pp.267-280
• /
• 2020

Mirror is a physical reflective surface, typically of glass coated with a metal amalgam, and it is to reflect an image clearly. They are available everywhere anytime and become an essential tool for us to observe our faces and appearances. With the advent of modern software technology, we are motivated to enhance the reflection capability of mirrors with the convenience and intelligence of realtime processing, microservices, and machine learning. In this paper, we present a development of Digital Mirror System that provides the realtime reflection functionality as mirror while providing additional convenience and intelligence including personal information retrieval, public information retrieval, appearance age detection, and emotion detection. Moreover, it provides a multi-model user interface of touch-based, voice-based, and gesture-based. We present our design and discuss how it can be implemented with current technology to deliver the realtime mirror reflection while providing useful information and machine learning intelligence.

• Journal of Digital Convergence
• /
• v.16 no.11
• /
• pp.495-500
• /
• 2018

This study examined the factors that can enhance the continuity of user experience in multi - device environment. First of all, regarding the structural difference and continuity of tasks, functional differences such as OS difference according to the characteristics of cross media, use of mouse and touch gesture were found to interfere with continuity. To increase continuity, metaphor and ambience To increase relevance and visibility. In the continuity part of visual memory and cognition, familiarity was given by the identity and similarity of visual perception elements, and it was found that familiarity factors are closely related to continuity. Finally, for the continuity of the user experience, we can see that the visibility factors as well as the meaning and layout consistency of the information are factors for the continuity of the user experience. Based on this, it was found that familiarity, consistency, and correlation were significant influences on continuity dimension of user experience, but visibility did not have a significant effect on continuity when regression analysis was conducted as factors of familiarity, consistency, correlation and visibility.

• Journal of Intelligence and Information Systems
• /
• v.17 no.1
• /
• pp.53-69
• /
• 2011

Vision and voice-based technologies are commonly utilized for human-robot interaction. But it is widely recognized that the performance of vision and voice-based interaction systems is deteriorated by a large margin in the real-world situations due to environmental and user variances. Human users need to be very cooperative to get reasonable performance, which significantly limits the usability of the vision and voice-based human-robot interaction technologies. As a result, touch screens are still the major medium of human-robot interaction for the real-world applications. To empower the usability of robots for various services, alternative interaction technologies should be developed to complement the problems of vision and voice-based technologies. In this paper, we propose the use of accelerometer-based gesture interface as one of the alternative technologies, because accelerometers are effective in detecting the movements of human body, while their performance is not limited by environmental contexts such as lighting conditions or camera's field-of-view. Moreover, accelerometers are widely available nowadays in many mobile devices. We tackle the problem of classifying acceleration signal patterns of 26 English alphabets, which is one of the essential repertoires for the realization of education services based on robots. Recognizing 26 English handwriting patterns based on accelerometers is a very difficult task to take over because of its large scale of pattern classes and the complexity of each pattern. The most difficult problem that has been undertaken which is similar to our problem was recognizing acceleration signal patterns of 10 handwritten digits. Most previous studies dealt with pattern sets of 8~10 simple and easily distinguishable gestures that are useful for controlling home appliances, computer applications, robots etc. Good features are essential for the success of pattern recognition. To promote the discriminative power upon complex English alphabet patterns, we extracted 'motion trajectories' out of input acceleration signal and used them as the main feature. Investigative experiments showed that classifiers based on trajectory performed 3%~5% better than those with raw features e.g. acceleration signal itself or statistical figures. To minimize the distortion of trajectories, we applied a simple but effective set of smoothing filters and band-pass filters. It is well known that acceleration patterns for the same gesture is very different among different performers. To tackle the problem, online incremental learning is applied for our system to make it adaptive to the users' distinctive motion properties. Our system is based on instance-based learning (IBL) where each training sample is memorized as a reference pattern. Brute-force incremental learning in IBL continuously accumulates reference patterns, which is a problem because it not only slows down the classification but also downgrades the recall performance. Regarding the latter phenomenon, we observed a tendency that as the number of reference patterns grows, some reference patterns contribute more to the false positive classification. Thus, we devised an algorithm for optimizing the reference pattern set based on the positive and negative contribution of each reference pattern. The algorithm is performed periodically to remove reference patterns that have a very low positive contribution or a high negative contribution. Experiments were performed on 6500 gesture patterns collected from 50 adults of 30~50 years old. Each alphabet was performed 5 times per participant using $Nintendo{(R)}$ $Wii^{TM}$ remote. Acceleration signal was sampled in 100hz on 3 axes. Mean recall rate for all the alphabets was 95.48%. Some alphabets recorded very low recall rate and exhibited very high pairwise confusion rate. Major confusion pairs are D(88%) and P(74%), I(81%) and U(75%), N(88%) and W(100%). Though W was recalled perfectly, it contributed much to the false positive classification of N. By comparison with major previous results from VTT (96% for 8 control gestures), CMU (97% for 10 control gestures) and Samsung Electronics(97% for 10 digits and a control gesture), we could find that the performance of our system is superior regarding the number of pattern classes and the complexity of patterns. Using our gesture interaction system, we conducted 2 case studies of robot-based edutainment services. The services were implemented on various robot platforms and mobile devices including $iPhone^{TM}$. The participating children exhibited improved concentration and active reaction on the service with our gesture interface. To prove the effectiveness of our gesture interface, a test was taken by the children after experiencing an English teaching service. The test result showed that those who played with the gesture interface-based robot content marked 10% better score than those with conventional teaching. We conclude that the accelerometer-based gesture interface is a promising technology for flourishing real-world robot-based services and content by complementing the limits of today's conventional interfaces e.g. touch screen, vision and voice.

• Design Convergence Study
• /
• v.15 no.1
• /
• pp.135-150
• /
• 2016

Due to the development of communication technology and expansion of Intelligent Transport System (ITS), the car is changing from a simple mechanical device to second living space which has comprehensive convenience function and is evolved into the platform which is playing as an interface for this role. As the interface area to provide various information to the passenger is being expanded, the research importance about smart car based user experience is rising. This study has a research objective to propose the guidelines regarding the smart car user experience elements. In order to conduct this study, smart car user experience elements were defined as function, interaction, and surface and through the discussions of UX/UI experts, 8 representative techniques, 14 representative techniques, and 8 locations of the glass windows were specified for each element. Following, the smart car users' priorities of the experience elements, which were defined through targeting 100 drivers, were analyzed in the form of questionnaire survey. The analysis showed that the users' priorities in applying the main techniques were in the order of safety, distance, and sensibility. The priorities of the production method were in the order of voice recognition, touch, gesture, physical button, and eye tracking. Furthermore, regarding the glass window locations, users prioritized the front of the driver's seat to the back. According to the demographic analysis on gender, there were no significant differences except for two functions. Therefore this showed that the guidelines of male and female can be commonly applied. Through user requirement analysis about individual elements, this study provides the guides about the requirement in each element to be applied to commercialized product with priority.