• Proceedings of the IEEK Conference
• /
• 1998.10a
• /
• pp.841-844
• /
• 1998

Korean Text to Sign Language Traslator could be applied to learn letters for both the deaf and hard-of-hearing people, and to have a conversation with normal people. This paper describes some useful dictionaries for developing korean text to sign language translator; Base sign language dictionary, Compound sign language dictionary, and Resemble sign language dictionary. As korean sign language is composed entirely of about 6,000 words, the additional dictionaries are required for matching them to korean written language. We design base sign language dictionary which was composed of basic symbols and moving picture of korean sign language, and propose the definition of compound isng language dictionary which was composed of symbols of base sing language. In addition, resemble sign language dictionary offer sign symbols and letters which is used same meaning in conversation. By using these methods, we could search quickly sign language during korean text to sign language translating process, and save storage space. We could also solve the lack of sign language words by using them, which are appeared on translating process.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.16 no.6
• /
• pp.1355-1362
• /
• 2021

Recently, not only congenital hearing impairment, but also the number of people with hearing impairment due to acquired factors is increasing. The environment in which sign language can be learned is poor. Therefore, this study intends to present a sign language (sign language number/sign language text) evaluation system as a sign language learning assistance tool for sign language learners. Therefore, in this paper, sign language is captured as an image using OpenCV and Convolutional Neural Network (CNN). In addition, we study a system that recognizes sign language behavior using MediaPipe, converts the meaning of sign language into text-type data, and provides it to users. Through this, self-directed learning is possible so that learners who learn sign language can judge whether they are correct dez. Therefore, we develop a sign language learning assistance system that helps us learn sign language. The purpose is to propose a sign language learning assistance system as a way to support sign language learning, the main language of communication for the hearing impaired.

• The Transactions of the Korea Information Processing Society
• /
• v.7 no.3
• /
• pp.756-765
• /
• 2000

In this paper, a korean text to sign language translation system is designed and implemented for the hearing impaired people to learn letters and to have a conversation with normal people. We adopt the direct method for machine translation which uses morphological analysis and the dictionary search. And we define the necessary sign language dictionaries. Based on this processes, the system translate korean sentences to sign language moving picture. The proposed dictionaries are composed of the basic sign language dictionary, the compound sing language dictionary, and the resemble sign language dictionary. The basic sign language dictionary includes basic symbols and moving pictures of korean sign language. The compound sing language dictionary is composed of key-words of basic sign language. In addition, we offered the similar letters at the resemble sign language dictionary. The moving pictures of searched sign symbols are displayed on a screen in GIF formats by continuous motion of sign symbols or represented by the finger spelling based on the korean code analysis. The proposed system can provide quick sign language search and complement the lack of sign languages in the translation process by using the various sign language dictionaries which are characterized as korean sign language. In addition, to represent the sign language using GIF makes it possible to save the storage space of the sign language. In addition, to represent the sign language using GIF makes it possible to save storage space of the sign language dictionary.

• The Journal of the Korea Contents Association
• /
• v.21 no.5
• /
• pp.508-524
• /
• 2021

Innovative translation circumstances encouraged by scientific technique have become an element that increases realization and possibility of expanding sign language translation and Korean sign language translation is facing a new challenge and opportunity. This study raises following questions and search for answers. First, when and how did Korean sign language translation appear in the course of the historical changes in Korean sign language? Second, what is the form and characteristic of translation produced as a result of Korean sign language translation? Third, what is the present condition and prospect of Korean sign language translation? Accordingly, this study examined how Korean sign language translation was formed historically and the form and characteristics of Korean sign language translation using integrated literature review method. As a result of the study, first, the form and characteristics of Korean sign language translation classified according to the historical transition process into latent phase, formation phase, and expansion phase were revealed. Second, the forms and characteristics of Korean sign language translation according to the Korean sign language corpus project and machine translation were derived. In addition, it apprehends its present condition and proposes its future prospect.

• KIPS Transactions on Software and Data Engineering
• /
• v.11 no.8
• /
• pp.325-330
• /
• 2022

This paper conducts the collection and experiment of datasets for deep learning research on sign language such as sign language recognition, sign language translation, and sign language segmentation for Korean sign language. There exist difficulties for deep learning research of sign language. First, it is difficult to recognize sign languages since they contain multiple modalities including hand movements, hand directions, and facial expressions. Second, it is the absence of training data to conduct deep learning research. Currently, KETI dataset is the only known dataset for Korean sign language for deep learning. Sign language datasets for deep learning research are classified into two categories: Isolated sign language and Continuous sign language. Although several foreign sign language datasets have been collected over time. they are also insufficient for deep learning research of sign language. Therefore, we attempted to collect a large-scale Korean sign language dataset and evaluate it using a baseline model named TSPNet which has the performance of SOTA in the field of sign language translation. The collected dataset consists of a total of 11,402 image and text. Our experimental result with the baseline model using the dataset shows BLEU-4 score 3.63, which would be used as a basic performance of a baseline model for Korean sign language dataset. We hope that our experience of collecting Korean sign language dataset helps facilitate further research directions on Korean sign language.

• Journal of Korea Multimedia Society
• /
• v.25 no.10
• /
• pp.1535-1545
• /
• 2022

Sign language has completely different meaning depending on the direction of the hand or the change of facial expression even with the same gesture. In this respect, it is crucial to capture the spatial-temporal structure information of each movement. However, sign language translation studies based on Sign2Gloss2Text only convey comprehensive spatial-temporal information about the entire sign language movement. Consequently, detailed information (facial expression, gestures, and etc.) of each movement that is important for sign language translation is not emphasized. Accordingly, in this paper, we propose Spatial-temporal Keypoints Centered Sign2Gloss2Text Translation, named STKC-Sign2 Gloss2Text, to supplement the sequential and semantic information of keypoints which are the core of recognizing and translating sign language. STKC-Sign2Gloss2Text consists of two steps, Spatial Keypoints Embedding, which extracts 121 major keypoints from each image, and Temporal Keypoints Embedding, which emphasizes sequential information using Bi-GRU for extracted keypoints of sign language. The proposed model outperformed all Bilingual Evaluation Understudy(BLEU) scores in Development(DEV) and Testing(TEST) than Sign2Gloss2Text as the baseline, and in particular, it proved the effectiveness of the proposed methodology by achieving 23.19, an improvement of 1.87 based on TEST BLEU-4.

• The Journal of the Korea Contents Association
• /
• v.21 no.9
• /
• pp.543-556
• /
• 2021

The purpose of this study is to examine possible improvements of operation of sign language interpreter qualification system through looking into specific perceptions of sign language interpreters and Deaf sign language interpreters in the field and Deaf people who use interpretation service on current sign language interpreter qualification system through focus group interview. As a result of the thematic analysis, three high-level categories were derived for each group, and as a result of recombining them, 'change to a qualification system in which sign language interpreting competency and expertise can be recognized' and 'improvement to a qualification system based on Korean Sign Language and Deaf culture' been sorted out. Based on the above research results, 'establishment of a permanent institution in charge of management and operation of sign language interpreter qualification tests', 'reorganization into a qualification system that reflects the professionalism, level and experience of sign language interpreters', 'all test questions and exam procedures are conducted mainly in sign language', 'a system in which the Deaf take the lead', and 'enactment of a code of ethics for sign language interpreters and Deaf interpreters' were suggested.

• The Journal of the Korea Contents Association
• /
• v.21 no.12
• /
• pp.505-516
• /
• 2021

The purpose of this study is to develop sign language interpreting content in the medical setting that facilitates Deaf people's access to medical services in situations where professional and accurate medical sign language interpreting is insufficient. To achieve the purpose of this study, we conducted a literature review, individual interviews for Deaf people, on-site requirement surveys of sign language interpreters and sign language experts, and medical and sign language expert consultations. Based on this, we developed sign language interpreting content such as main care contextual scenarios, basic medical terms, and medical term descriptions. Through this study, we developed medical sign language content considering the situation and medical importance of Deaf people to promote expertise in the medial sign language area and developed a responsive website of sign language medical dictionary that effectively and efficiently delivers information to Deaf people and sign language interpreters; we realized the need and importance of sign language translation for Deaf people to be the main bodies.

• Journal of the Ergonomics Society of Korea
• /
• v.34 no.5
• /
• pp.411-426
• /
• 2015

Objective: The goal of this thesis is to design the interaction structure and framework of system to recognize sign language. Background: The sign language of meaningful individual gestures is combined to construct a sentence, so it is difficult to interpret and recognize the meaning of hand gesture for system, because of the sequence of continuous gestures. This being so, in order to interpret the meaning of individual gesture correctly, the interaction structure and framework are needed so that they can segment the indication of individual gesture. Method: We analyze 700 sign language words to structuralize the sign language gesture interaction. First of all, we analyze the transformational patterns of the hand gesture. Second, we analyze the movement of the transformational patterns of the hand gesture. Third, we analyze the type of other gestures except hands. Based on this, we design a framework for sign language interaction. Results: We elicited 8 patterns of hand gesture on the basis of the fact on whether the gesture has a change from starting point to ending point. And then, we analyzed the hand movement based on 3 elements: patterns of movement, direction, and whether hand movement is repeating or not. Moreover, we defined 11 movements of other gestures except hands and classified 8 types of interaction. The framework for sign language interaction, which was designed based on this mentioned above, applies to more than 700 individual gestures of the sign language, and can be classified as an individual gesture in spite of situation which has continuous gestures. Conclusion: This study has structuralized in 3 aspects defined to analyze the transformational patterns of the starting point and the ending point of hand shape, hand movement, and other gestures except hands for sign language interaction. Based on this, we designed the framework that can recognize the individual gestures and interpret the meaning more accurately, when meaningful individual gesture is input sequence of continuous gestures. Application: When we develop the system of sign language recognition, we can apply interaction framework to it. Structuralized gesture can be used for using database of sign language, inventing an automatic recognition system, and studying on the action gestures in other areas.

• Journal of Convergence for Information Technology
• /
• v.11 no.5
• /
• pp.17-22
• /
• 2021

It is difficult for the hearing impaired to use the counseling service without sign language interpretation. Due to the shortage of sign language interpreters, it takes a lot of time to connect to sign language interpreters, or there are many cases where the connection is not available. Therefore, in this paper, we propose a system that captures sign language as an image using OpenCV and CNN (Convolutional Neural Network), recognizes sign language motion, and converts the meaning of sign language into textual data and provides it to users. The counselor can conduct counseling by reading the stored sign language translation counseling contents. Consultation is possible without a professional sign language interpreter, reducing the burden of waiting for a sign language interpreter. If the proposed system is applied to counseling services for the hearing impaired, it is expected to improve the effectiveness of counseling and promote academic research on counseling for the hearing impaired in the future.