• Journal of Korea Multimedia Society
• /
• v.25 no.10
• /
• pp.1535-1545
• /
• 2022

Sign language has completely different meaning depending on the direction of the hand or the change of facial expression even with the same gesture. In this respect, it is crucial to capture the spatial-temporal structure information of each movement. However, sign language translation studies based on Sign2Gloss2Text only convey comprehensive spatial-temporal information about the entire sign language movement. Consequently, detailed information (facial expression, gestures, and etc.) of each movement that is important for sign language translation is not emphasized. Accordingly, in this paper, we propose Spatial-temporal Keypoints Centered Sign2Gloss2Text Translation, named STKC-Sign2 Gloss2Text, to supplement the sequential and semantic information of keypoints which are the core of recognizing and translating sign language. STKC-Sign2Gloss2Text consists of two steps, Spatial Keypoints Embedding, which extracts 121 major keypoints from each image, and Temporal Keypoints Embedding, which emphasizes sequential information using Bi-GRU for extracted keypoints of sign language. The proposed model outperformed all Bilingual Evaluation Understudy(BLEU) scores in Development(DEV) and Testing(TEST) than Sign2Gloss2Text as the baseline, and in particular, it proved the effectiveness of the proposed methodology by achieving 23.19, an improvement of 1.87 based on TEST BLEU-4.

• The KIPS Transactions:PartB
• /
• v.14B no.2
• /
• pp.107-118
• /
• 2007

This paper proposes a 3D avatar which expresses sign language in a very using lips, facial expression, complexion, pupil motion and body motion as well as hand shape, Hand posture and hand motion to overcome the limitation of conventional sign language avatars from a deaf's viewpoint. To describe motion data of hand and other body components structurally and enhance the performance of databases, we introduce the concept of a hyper sign sentence. We show the superiority of the developed system by a usability test through a questionnaire survey.

• Proceedings of the IEEK Conference
• /
• 1999.06a
• /
• pp.651-654
• /
• 1999

In this paper we propose an emotional expression method using a comic model and special marks for effective sign-language communications. Until now we have investigated to produce more realistic facial and emotional expression. When representing only emotional expression, however, a comic expression could be better than the real picture of a face. The comic face is a comic-style expression model in which almost components except the necessary parts like eyebrows, eyes, nose and mouth are discarded. In the comic model, we can use some special marks for the purpose of emphasizing various emotions. We represent emotional expression using Action Units(AU) of Facial Action Coding System(FACS) and define Special Unit(SU) for emphasizing the emotions. Experimental results show a possibility that the proposed method could be used efficiently for sign-language image communications.

• Journal of Korean Theatre Studies Association
• /
• no.52
• /
• pp.271-317
• /
• 2014

This essay studied the acting theories of Diderot, Delsarte, Meyerhold, and Artaud to know the historical formation of 'sign acting' and its theoretical and aesthetic appropriateness. The sign acting so far discussed shows the repetitive patterns of idea as follows. The sign acting (1) emphasizes the physical expression such as gesture and movement, (2) assumes that the physical expression functions as a sign evoking special emotion and thought, (3) thus recommends the imitation of the outer sign, (4) uses a tableau for the effective reception of outer sign, (5) aims for the spectator oriented aesthetics as it stresses the result of outer sign rather than the creative process of a role, (6) assumes that the emotional reaction or the intellectual understanding springs from the physical experience, (7) thus emphasizes the physical language rather than speech, (8) can attain the appropriateness of physical language by the recent theories of cognitive science. Besides having such commonness, the sign acting also reveals the individual differences. For instance, the intended sign for Diderot and Delsarte was the sign of emotion, for Meyerhold the stylized sign of circus and acrobatics, and for Artaud the spiritual sign. If Diderot and Meyerhold demands the cool consciousness for the correct sign acting, Artaud's sign acting tends to pursue the state of trance. And if Diderot, Delsarte, and Meyerhold think the sign acting on the level of sensory appeal, Artaud insists that the sign acting should dismantle the spectator's sense. As such the discussion of sign acting shows both recurrent ideas and new visions, forming an unity out of diversity. Perhaps the sign acting is a matter of practice before we consider it as a theory. It is not only supposed to have been existed practically since ancient theatre, but also used by actors consciously and unconsciously in expressing certain emotion and thought. We need to study the sign acting more academically, considering its long history and aesthetic potentials. In fact the sign acting has been an essential element of acting, in spite of bad reputation judging it as a banal and worn-out style. It is true that the sign acting, in the worst case, could produce a stereotypical expression. It was this aspect of sign acting that caused a fierce negative reaction of the realists who sought the natural expression based upon psychological truth. Of course the sign acting has a serious problem when it stays banal and artificial. But we need to see this issue from a different perspective. What is the natural expression of emotion? How is it free from the learned way of expression? In some respect, we use, in reality, a learned expression of emotion that could be accepted socially. For instance, when we attend a funeral, we use the outer sign of mourning gestures learned socially. If a semiotic expression pervades various aspects of our life, the acting, being the representation of life, seems not to be free from codified expression. The sign acting could be used consciously and unconsciously in all kinds of acting.

• Language and Information
• /
• v.10 no.1
• /
• pp.71-91
• /
• 2006

The linguistic characteristics of a sign language provide an important clue for an automatic construction of its expression from a given natural language sentence. For such characterization, we focus on the identification of elided constituents, the mapping of property-changing information into spatio-temporal dimension, and the need for rearranging the order of component information for enhanced quality of delivery. We use our characterization to implement a system that converts sentences in Korean into corresponding expressions in the Korean Sign Language.

• Proceedings of the IEEK Conference
• /
• 1999.11a
• /
• pp.536-539
• /
• 1999

In this paper we propose a 3D emotional expression method using a comic model for effective sign-language communications. Until now we have investigated to produce more realistic facial and emotional expression. When representing only emotional expression, however, a comic expression could be better than the real picture of a face. The comic face is a comic-style expression model in which almost components except the necessary parts like eyebrows, eyes, nose and mouth are discarded. We represent emotional expression using Action Units(AU) of Facial Action Coding System(FACS). Experimental results show a possibility that the proposed method could be used efficiently for sign-language image communications.

• Proceedings of the Korean Institute of Interior Design Conference
• /
• 2007.05a
• /
• pp.176-180
• /
• 2007

The stage space in theatre for the performance of a play requires two aspects of the physical space for a play and the image as the background or the theme of a play. This research focuses on the scene design that creates the background or the theme of a play and communicates it to the audience. The scene design is composed of various design elements of space and objects as the properties of a play. Design elements and the objects are the design languages in various form; plane, three-dimensional, multi-dimensional form. These design language have the significant meaning as a sign like human language. The scene design is completed with rhetorical expression for communication in theatre. This research defines the category of meaning that design elements of scene design can create and the rhetorical expression of the scene design language. King Lear directed by Robert G. Anderson was analyzed with the category of design elements as a sign and the pattern of the rhetorical expression. The scene design for a play is completed effectively by the rhetorical expression of design elements as the design language for communication with the audience in theatre.

• The Research Journal of the Costume Culture
• /
• v.10 no.3
• /
• pp.213-235
• /
• 2002

The purpose of this study was to recognize the meanings of plastic language by analyzing the sign of the Graffiti depicted in western modern costumes. And the scope of this study was focused on analysing the costumes from 1980's when the Graffiti was recognized as one of the plastic arts. Graffiti was an unprofessional and covert desire of self-expression, having a close relation with our lives. It was used in diverse ways with repeating creation, development, and extinction, from expressing liveliness in Old Stone Age through describing symbolic meaning in the modern art. Graffiti means rude, humorous, or political writing and pictures on the walls of buildings. It's different from the delicate letters or pictures to inscript on the tree or rocks. Being introduced as part of Post-modernism in the 1970s, Graffiti was acknowledged as a new artistic action with the culture of hip-hop. In addition the Graffiti, the expression of sign was reflected artist's internal consciousness with boundless sign. The sign is something to transmit message from the nonverbal point of view, the oldest sign was the sign for petition or the expression of something as a primitive form. Sign can be defined as a framework of life inherited as a practice from the primitive age long past. Graffiti was cited as a concrete example of sign, theme of this paper, and general understanding on sign expression was pursued through free work which all artists present with essential and positive ways using signs, their own languages, and life style. The result of this study was summarized as follows: Since the end of the 20th century, a lot of fashion designers have represented symbolic images, such as letter and signs, as purely personal ways of expression through the western costumes. It was found that fashion designers of the Graffiti whose free work style from inherent inner consciousness might provide a basic framework to search for complex signs of modem costumes naturally understood life itself as plastic art, and sublimated human inherent desire and their inner world through their works.

• International Journal of Contents
• /
• v.7 no.1
• /
• pp.14-22
• /
• 2011

Temporal Sign Language Recognition (TSLR) from hand motion is an active area of gesture recognition research in facilitating efficient communication with deaf people. TSLR systems consist of two stages: a motion sensing step which extracts useful features from signers' motion and a classification process which classifies these features as a performed sign. This work focuses on two of the research problems, namely unknown time varying signal of sign languages in feature extraction stage and computing complexity and time consumption in classification stage due to a very large sign sequences database. In this paper, we propose a combination of Dynamic Time Warping (DTW) and application of the Single hidden Layer Feedforward Neural networks (SLFNs) trained by Extreme Learning Machine (ELM) to cope the limitations. DTW has several advantages over other approaches in that it can align the length of the time series data to a same prior size, while ELM is a useful technique for classifying these warped features. Our experiment demonstrates the efficiency of the proposed method with the recognition accuracy up to 98.67%. The proposed approach can be generalized to more detailed measurements so as to recognize hand gestures, body motion and facial expression.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.21 no.2
• /
• pp.461-467
• /
• 2017

As the number of smart devices increases, e-book contents and services are proliferating. However, the text based e-book is difficult for a hearing-impairment person to understand. In this paper, we developed an android based application in which we can choose an e-book text file and each sentence is translated to sign-language elements which are shown in videos that are retrieved from the sign-language contents server. We used the korean sentence to sign-language translation algorithm based on the morpheme analysis. The proposed translation algorithm consists of 3 stages. Firstly, some elements in a sentence are removed for typical sign-language usages. Secondly, the tense of the sentence and the expression alteration are applied. Finally, the honorific forms are considered and word positions in the sentence are revised. We also proposed a new method to evaluate the performance of the translation algorithm and demonstrated the superiority of the algorithm through the translation results of 100 reference sentences.