• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2009.01a
• /
• pp.425-428
• /
• 2009

We have researched and developed the caricature generation system PICASSO. PICASSO outputs the deformed facial caricature by comparing input face with prepared mean face. We specialized it as PICASSO-2 for exhibiting a robot at Aichi EXPO2005. This robot enforced by PICASSO-2 drew a facial caricature on the shrimp rice cracker with the laser pen. We have been recently exhibiting another revised robot characterized by a brush drawing. This system takes a couple of facial images with CCD camera, extracts the facial features from the images, and generates the facial caricature in real time. We experimentally evaluated the performance of the caricatures using a lot of data taken in Aichi EXPO2005. As a result it was obvious that this system were not sufficient in accuracy of eyebrow region extraction and mouth detection. In this paper, we propose the improved methods for eyebrow region extraction and mouth detection.

• Transactions of the Korean Society of Mechanical Engineers
• /
• v.18 no.9
• /
• pp.2349-2357
• /
• 1994

This paper presents a facial expression transformation algorithm and drawing rule generation algolithm for a portrait drawing robot which was developed for the '93 Taejeon EXPO. The developed algorithm was mainly focused on the robust automatic generation of robot programs with the consideration that the drawing robot should work without any limitation of the age, sex or race for the persons. In order to give more demonstratin effects, the facial expression change of the pictured person was performed.

• Journal of Institute of Control, Robotics and Systems
• /
• v.12 no.3
• /
• pp.255-263
• /
• 2006

In the last decade, face analysis, e.g. face detection, face recognition, facial expression recognition, is a very lively and expanding research field. As computer animated agents and robots bring a social dimension to human computer interaction, interest in this research field is increasing rapidly. In this paper, we introduce an artificial emotion mimic system which can recognize human facial expressions and also generate the recognized facial expression. In order to recognize human facial expression in real-time, we propose a facial expression classification method that is performed by weak classifiers obtained by using new rectangular feature types. In addition, we make the artificial facial expression using the developed robotic system based on biological observation. Finally, experimental results of facial expression recognition and generation are shown for the validity of our robotic system.

• 제어로봇시스템학회:학술대회논문집
• /
• 2004.08a
• /
• pp.1229-1234
• /
• 2004

Face robots capable of expressing their emotional status, can be adopted as an e cient tool for friendly communication between the human and the machine. In this paper, we present a face robot actuated with arti cial muscle based on dielectric elastomer. By exploiting the properties of polymers, it is possible to actuate the covering skin, and provide human-like expressivity without employing complicated mechanisms. The robot is driven by seven types of actuator modules such as eye, eyebrow, eyelid, brow, cheek, jaw and neck module corresponding to movements of facial muscles. Although they are only part of the whole set of facial motions, our approach is su cient to generate six fundamental facial expressions such as surprise, fear, angry, disgust, sadness, and happiness. Each module communicates with the others via CAN communication protocol and according to the desired emotional expressions, the facial motions are generated by combining the motions of each actuator module. A prototype of the robot has been developed and several experiments have been conducted to validate its feasibility.

• Proceedings of the KIEE Conference
• /
• 2004.11c
• /
• pp.147-149
• /
• 2004

There are no authentic solutions in a face region extraction problem though it is an important part of pattern recognition and has diverse application fields. It is not easy to develop the facial region extraction algorithm because the facial image is very sensitive according to age, sex, and illumination. In this paper, to solve these difficulties, a fuzzy color filer based on the facial region extraction algorithm is proposed. The fuzzy color filter makes the robust facial region extraction enable by modeling the skin color. Especially, it is robust in facial region extraction with various illuminations. In addition, to identify the fuzzy color filter, a linear matrix inequality(LMI) optimization method is used. Finally, the simulation result is given to confirm the superiority of the proposed algorithm.

• The Journal of Korea Robotics Society
• /
• v.8 no.2
• /
• pp.104-115
• /
• 2013

For natural human-robot interaction, we need to know location and shape of facial feature in real environment. In order to track facial feature robustly, we can use the method combining particle filter and active appearance model. However, processing speed of this method is too slow. In this paper, we propose two ideas to improve efficiency of this method. The first idea is changing the number of particles situationally. And the second idea is switching the prediction model situationally. Experimental results is presented to show that the proposed method is about three times faster than the method combining particle filter and active appearance model, whereas the performance of the proposed method is maintained.

• Journal of Institute of Control, Robotics and Systems
• /
• v.11 no.1
• /
• pp.50-57
• /
• 2005

We present a simple and effective method of face and facial feature detection under pose variation of user face in complex background for the human-robot interaction. Our approach is a flexible method that can be performed in both color and gray facial image and is also feasible for detecting facial features in quasi real-time. Based on the characteristics of the intensity of neighborhood area of facial features, new directional template for facial feature is defined. From applying this template to input facial image, novel edge-like blob map (EBM) with multiple intensity strengths is constructed. Regardless of color information of input image, using this map and conditions for facial characteristics, we show that the locations of face and its features - i.e., two eyes and a mouth-can be successfully estimated. Without the information of facial area boundary, final candidate face region is determined by both obtained locations of facial features and weighted correlation values with standard facial templates. Experimental results from many color images and well-known gray level face database images authorize the usefulness of proposed algorithm.

• 제어로봇시스템학회:학술대회논문집
• /
• 2003.10a
• /
• pp.783-788
• /
• 2003

In this paper, we present a new approach to detect and recognize human face in the image from vision camera equipped on the mobile robot platform. Due to the mobility of camera platform, obtained facial image is small and pose-various. For this condition, new algorithm should cope with these constraints and can detect and recognize face in nearly real time. In detection step, ‘coarse to fine’ detection strategy is used. Firstly, region boundary including face is roughly located by dual ellipse templates of facial color and on this region, the locations of three main facial features- two eyes and mouth-are estimated. For this, simplified facial feature maps using characteristic chrominance are made out and candidate pixels are segmented as eye or mouth pixels group. These candidate facial features are verified whether the length and orientation of feature pairs are suitable for face geometry. In recognition step, pseudo-convex hull area of gray face image is defined which area includes feature triangle connecting two eyes and mouth. And random lattice line set are composed and laid on this convex hull area, and then 2D appearance of this area is represented. From these procedures, facial information of detected face is obtained and face DB images are similarly processed for each person class. Based on facial information of these areas, distance measure of match of lattice lines is calculated and face image is recognized using this measure as a classifier. This proposed detection and recognition algorithms overcome the constraints of previous approach [15], make real-time face detection and recognition possible, and guarantee the correct recognition irregardless of some pose variation of face. The usefulness at mobile robot application is demonstrated.

• The Journal of the Korea Contents Association
• /
• v.15 no.2
• /
• pp.74-85
• /
• 2015

The positive effect of percussion instruments can induce increases in self-esteem and decreases in depression in the elderly. Based on this, the content for a percussion instrument robot that the elderly can use to play music is developed. The elements of the interaction between the elderly and the robot through the robot content are extracted. Music that arouses positive memories in the elderly is selected as part of the music therapy robot content in order to relieve depression, and a scoring system for playing music is constructed. In addition, the interaction components of the robot's facial expressions, which stimulate emotions and sensitivity in the elderly, are also designed. These components enable the elderly to take an active part in using the instrument to change the robot's facial expressions, which have three degrees of emotion: neutral-happy, happy, and very happy. The robot is not only a music game machine: it also maximizes the relief of depression in the elderly through interactions with the robot that allow the elderly person to listen to what the robot plays and through the elderly person becoming involved and playing music along with the robot.

• Journal of Institute of Control, Robotics and Systems
• /
• v.22 no.3
• /
• pp.241-246
• /
• 2016

Nowadays many people have an interest in facial expression and the behavior of people. These are human-robot interaction (HRI) researchers utilize digital image processing, pattern recognition and machine learning for their studies. Facial feature point detector algorithms are very important for face recognition, gaze tracking, expression, and emotion recognition. In this paper, a cascade facial feature point detector is used for finding facial feature points such as the eyes, nose and mouth. However, the detector has difficulty extracting the feature points from several images, because images have different conditions such as size, color, brightness, etc. Therefore, in this paper, we propose an algorithm using a modified cascade facial feature point detector using a convolutional neural network. The structure of the convolution neural network is based on LeNet-5 of Yann LeCun. For input data of the convolutional neural network, outputs from a cascade facial feature point detector that have color and gray images were used. The images were resized to $32{\times}32$. In addition, the gray images were made into the YUV format. The gray and color images are the basis for the convolution neural network. Then, we classified about 1,200 testing images that show subjects. This research found that the proposed method is more accurate than a cascade facial feature point detector, because the algorithm provides modified results from the cascade facial feature point detector.