• Journal of Institute of Control, Robotics and Systems
• /
• v.18 no.4
• /
• pp.328-336
• /
• 2012

In this paper, we propose a robust fingertip extraction and extended Continuously Adaptive Mean Shift (CAMSHIFT) based robust hand gesture recognition for natural human-like HRI (Human-Robot Interaction). Firstly, for efficient and rapid hand detection, the hand candidate regions are segmented by the combination with robust $YC_bC_r$ skin color model and haar-like features based adaboost. Using the extracted hand candidate regions, we estimate the palm region and fingertip position from distance transformation based voting and geometrical feature of hands. From the hand orientation and palm center position, we find the optimal fingertip position and its orientation. Then using extended CAMSHIFT, we reliably track the 2D hand gesture trajectory with extracted fingertip. Finally, we applied the conditional density propagation (CONDENSATION) to recognize the pre-defined temporal motion trajectories. Experimental results show that the proposed algorithm not only rapidly extracts the hand region with accurately extracted fingertip and its angle but also robustly tracks the hand under different illumination, size and rotation conditions. Using these results, we successfully recognize the multiple hand gestures.

• 제어로봇시스템학회:학술대회논문집
• /
• 2005.06a
• /
• pp.1938-1943
• /
• 2005

Network human-robot interface is an important research topic. In home application, users access the robotic system directly via voice, gestures or through the network. Users explore a system by using the services provided by this system and to some extend users are enable to participate in a service as partners. A service may be provided by a robot, a group of robots or robots and other network connected systems (distributed sensors, information systems, etc). All these services are done in the network environment, where uncertainty such as the unstable network connection, the availability of the partners in a service, exists. Moreover, these services are controlled by several users, accessing at different time by different methods. Our research aimed at solving this problem to provide a high available level, flexible coordination system. In this paper, a multi-agent framework is proposed. This framework is validated by using our new concept of slave agents, a responsive multi-agent environment, a virtual directory facilitator (VDF), and a task allocation system using contract net protocol. Our system uses a mixed model between distributed and centralized model. It uses a centralized agent management system (AMS) to control the overall system. However, the partners and users may be distributed agents connected to the center through agent communication or centralized at the AMS container using the slave agents to represent the physical agents. The system is able to determine the task allocation for a group of robot working as a team to provide a service. A number of experiments have been conducted successfully in our lab environment using Issac robot, a PDA for user agent and a wireless network system, operated under our multi agent framework control. The experiments show that this framework works well and provides some advantages to existing systems.

• Journal of Intelligence and Information Systems
• /
• v.17 no.1
• /
• pp.53-69
• /
• 2011

Vision and voice-based technologies are commonly utilized for human-robot interaction. But it is widely recognized that the performance of vision and voice-based interaction systems is deteriorated by a large margin in the real-world situations due to environmental and user variances. Human users need to be very cooperative to get reasonable performance, which significantly limits the usability of the vision and voice-based human-robot interaction technologies. As a result, touch screens are still the major medium of human-robot interaction for the real-world applications. To empower the usability of robots for various services, alternative interaction technologies should be developed to complement the problems of vision and voice-based technologies. In this paper, we propose the use of accelerometer-based gesture interface as one of the alternative technologies, because accelerometers are effective in detecting the movements of human body, while their performance is not limited by environmental contexts such as lighting conditions or camera's field-of-view. Moreover, accelerometers are widely available nowadays in many mobile devices. We tackle the problem of classifying acceleration signal patterns of 26 English alphabets, which is one of the essential repertoires for the realization of education services based on robots. Recognizing 26 English handwriting patterns based on accelerometers is a very difficult task to take over because of its large scale of pattern classes and the complexity of each pattern. The most difficult problem that has been undertaken which is similar to our problem was recognizing acceleration signal patterns of 10 handwritten digits. Most previous studies dealt with pattern sets of 8~10 simple and easily distinguishable gestures that are useful for controlling home appliances, computer applications, robots etc. Good features are essential for the success of pattern recognition. To promote the discriminative power upon complex English alphabet patterns, we extracted 'motion trajectories' out of input acceleration signal and used them as the main feature. Investigative experiments showed that classifiers based on trajectory performed 3%~5% better than those with raw features e.g. acceleration signal itself or statistical figures. To minimize the distortion of trajectories, we applied a simple but effective set of smoothing filters and band-pass filters. It is well known that acceleration patterns for the same gesture is very different among different performers. To tackle the problem, online incremental learning is applied for our system to make it adaptive to the users' distinctive motion properties. Our system is based on instance-based learning (IBL) where each training sample is memorized as a reference pattern. Brute-force incremental learning in IBL continuously accumulates reference patterns, which is a problem because it not only slows down the classification but also downgrades the recall performance. Regarding the latter phenomenon, we observed a tendency that as the number of reference patterns grows, some reference patterns contribute more to the false positive classification. Thus, we devised an algorithm for optimizing the reference pattern set based on the positive and negative contribution of each reference pattern. The algorithm is performed periodically to remove reference patterns that have a very low positive contribution or a high negative contribution. Experiments were performed on 6500 gesture patterns collected from 50 adults of 30~50 years old. Each alphabet was performed 5 times per participant using $Nintendo{(R)}$ $Wii^{TM}$ remote. Acceleration signal was sampled in 100hz on 3 axes. Mean recall rate for all the alphabets was 95.48%. Some alphabets recorded very low recall rate and exhibited very high pairwise confusion rate. Major confusion pairs are D(88%) and P(74%), I(81%) and U(75%), N(88%) and W(100%). Though W was recalled perfectly, it contributed much to the false positive classification of N. By comparison with major previous results from VTT (96% for 8 control gestures), CMU (97% for 10 control gestures) and Samsung Electronics(97% for 10 digits and a control gesture), we could find that the performance of our system is superior regarding the number of pattern classes and the complexity of patterns. Using our gesture interaction system, we conducted 2 case studies of robot-based edutainment services. The services were implemented on various robot platforms and mobile devices including $iPhone^{TM}$. The participating children exhibited improved concentration and active reaction on the service with our gesture interface. To prove the effectiveness of our gesture interface, a test was taken by the children after experiencing an English teaching service. The test result showed that those who played with the gesture interface-based robot content marked 10% better score than those with conventional teaching. We conclude that the accelerometer-based gesture interface is a promising technology for flourishing real-world robot-based services and content by complementing the limits of today's conventional interfaces e.g. touch screen, vision and voice.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.8 no.7
• /
• pp.1480-1487
• /
• 2004

This study is proposed the novel gesture recognition method for the remote camera robot control. To recognize the dynamics gesture, the preprocessing step is the image segmentation. The conventional methods for the effectively object segmentation has need a lot of the cole. information about the object(hand) image. And these methods in the recognition step have need a lot of the features with the each object. To improve the problems of the conventional methods, this study proposed the novel method to recognize the dynamic hand gesture such as the MMS(Max-Min Search) method to segment the object image, MSM(Mean Space Mapping) method and COG(Conte. Of Gravity) method to extract the features of image, and the structure of recognition MLPNN(Multi Layer Perceptron Neural Network) to recognize the dynamic gestures. In the results of experiment, the recognition rate of the proposed method appeared more than 90[%], and this result is shown that is available by HCI(Human Computer Interface) device for .emote robot control.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 2008.04a
• /
• pp.400-402
• /
• 2008

The human-computer interaction technology (HCI) that has played an important role in the exchange of information between human being and computer belongs to a key field for information technology. Recently, control studies through which robots and control devices are controlled by using the movements of a person's body or hands without using conventional input devices such as keyboard and mouse, have been going only in diverse aspects, and their importance has been steadily increasing. This study is proposing a recognition method of user's gestures by applying measurements from an acceleration sensor to the PCA algorithm.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2014.10a
• /
• pp.231-232
• /
• 2014

We develop a remote control system of a walking robot using a Leap motion sensor. Hand gestures and the position of fingers are provided from the Leap motion sensor. We use Processing and the LeapMotionP5 library for the development software.

• 한국HCI학회:학술대회논문집
• /
• 2009.02a
• /
• pp.186-192
• /
• 2009

In this paper, we represent the recognition method for human frontal upper body pose. In HCI(Human Computer Interaction) and HRI(Human Robot Interaction) when a interaction is established the human has usually frontal direction to the robot or computer and use hand gestures then we decide to focus on human frontal upper-body pose, The two main difficulties are firstly human pose is consist of many parts which cause high DOF(Degree Of Freedom) then the modeling of human pose is difficult. Secondly the matching between image features and modeling information is difficult. Then using Pictorial Model we model the human main poses which are mainly took the space of frontal upper-body poses and we recognize the main poses by making main pose database. using determined main pose we used the model parameters for particle filter which predicts the posterior distribution for pose parameters and can determine more specific pose by updating model parameters from the particle having the maximum likelihood. Therefore based on recognizing main poses and tracking the specific pose we recognize the human frontal upper body poses.

• International Journal of Computer Science & Network Security
• /
• v.21 no.2
• /
• pp.110-119
• /
• 2021

The Coronavirus or COVID-19 is contagiousness virus that infected almost every single part of the world. This pandemic forced a major country did lockdown and stay at a home policy to reduce virus spread and the number of victims. Interactions between humans and robots form a popular subject of research worldwide. In medical robotics, the primary challenge is to implement natural interactions between robots and human users. Human communication consists of dynamic processes that involve joint attention and attracting each other. Coordinated care involves sharing among agents of behaviours, events, interests, and contexts in the world from time to time. The robotics arm is an expensive and complicated system because robot simulators are widely used instead of for rehabilitation purposes in medicine. Interaction in natural ways is necessary for disabled persons to work with the robot simulator. This article proposes a low-cost rehabilitation system by building an arm gesture tracking system based on a depth camera that can capture and interpret human gestures and use them as interactive commands for a robot simulator to perform specific tasks on the 3D block. The results show that the proposed system can help patients control the rotation and movement of the 3D arm using their hands. The pilot testing with healthy subjects yielded encouraging results. They could synchronize their actions with a 3D robotic arm to perform several repetitive tasks and exerting 19920 J of energy (kg.m².S^-2). The average of consumed energy mentioned before is in medium scale. Therefore, we relate this energy with rehabilitation performance as an initial stage and can be improved further with extra repetitive exercise to speed up the recovery process.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.23 no.6
• /
• pp.551-557
• /
• 2013

In intelligent robot research, hand gesture recognition has been an important issue. And techniques that recognize simple gestures are commercialized in smart phone, smart TV for swiping screen or volume control. For gesture recognition, robust hand detection is important and necessary but it is challenging because hand shape is complex and hard to be detected in cluttered background, variant illumination. In this paper, we propose efficient hand detection algorithm for detecting pointing hand for recognition of place where user pointed. To minimize false detections, ROIs are generated within the compact search region using skin color detection result. The ROIs are verified by HOG-SVM and pointing direction is computed by both detection results of head-shoulder and hand. In experiment, it is shown that proposed method shows good performance for hand detection.

• KIPS Transactions on Software and Data Engineering
• /
• v.5 no.2
• /
• pp.107-116
• /
• 2016

Industrial robots have been usually controlled using text-based programming languages provided by each manufacturer with its button-based TP(Teaching Pendent) terminal. Unfortunately, when we consider that people who manipulate TPs in manufacturing sites are mostly unskilled with no background knowledge about computer programming, these text-based programming languages using button-based interaction on manufacturing sites are too difficult for them to learn and use. In order to overcome the weaknesses of the text-based programming language, we propose a visual programming language that can be easily used on gesture-enabled devices. Especially, in our visual programming environment, each command is represented as a block and robots are controlled by stacking those blocks using drag-and-drop gestures, which is easily learnable even by beginners. In this paper, we utilize a widely-spread device, Tablet PC as the gesture-enabled TP. Considering that Tablet PC has limited display space in contrast to PC environments, we designed different kinds of sets of command blocks and conducted user tests. Based on the experiment results, we propose an effective set of command blocks for Tablet PC environment.