• The Journal of the Korea Contents Association
• /
• v.21 no.5
• /
• pp.508-524
• /
• 2021

Innovative translation circumstances encouraged by scientific technique have become an element that increases realization and possibility of expanding sign language translation and Korean sign language translation is facing a new challenge and opportunity. This study raises following questions and search for answers. First, when and how did Korean sign language translation appear in the course of the historical changes in Korean sign language? Second, what is the form and characteristic of translation produced as a result of Korean sign language translation? Third, what is the present condition and prospect of Korean sign language translation? Accordingly, this study examined how Korean sign language translation was formed historically and the form and characteristics of Korean sign language translation using integrated literature review method. As a result of the study, first, the form and characteristics of Korean sign language translation classified according to the historical transition process into latent phase, formation phase, and expansion phase were revealed. Second, the forms and characteristics of Korean sign language translation according to the Korean sign language corpus project and machine translation were derived. In addition, it apprehends its present condition and proposes its future prospect.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2020.07a
• /
• pp.456-458
• /
• 2020

한국수화언어(한국수어)는 농인들이 사용하는 언어이며, 농인이라 함은 청각장애를 가진 사람으로서 한국수어를 일상어로 사용하는 사람을 말한다. 수어를 하나의 언어로써 다른 언어로의 번역 또는 상호번역을 위하여 기계학습 기반의 기술이 연구개발 되고 있으나, 수어는 영상 기반의 언어이고 한국수어의 문법 및 사전체계의 구축이 진행 중인 이유로 한국수어의 번역기술은 상대적으로 다른 이종언어 간의 번역기술에 비하여 발전속도가 느리다. 본 논문에서는 한국어를 한국수어로 번역하여 표현하기 위하여 필요한 수어 스크립트 포맷 및 데이터 인터페이스 규격을 제안한다.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2024.01a
• /
• pp.317-319
• /
• 2024

본 논문에서는 비디오 캡셔닝 알고리즘을 적용한 수어 번역 및 행동 인식 알고리즘을 적용한 수어 인식 알고리즘을 제안한다. 본 논문에 사용된 비디오 캡셔닝 알고리즘으로 40개의 연속된 입력 데이터 프레임을 CNN 네트워크를 통해 임베딩 하고 트랜스포머의 입력으로 하여 문장을 출력하였다. 행동 인식 알고리즘은 랜덤 샘플링을 하여 한 영상에 40개의 인덱스에서 40개의 연속된 데이터에 CNN 네트워크를 통해 임베딩하고 GRU, 트랜스포머를 결합한 RNN 모델을 통해 인식 결과를 출력하였다. 수어 번역에서 BLEU-4의 경우 7.85, CIDEr는 53.12를 얻었고 수어 인식으로 96.26%의 인식 정확도를 얻었다.

• The Journal of Bigdata
• /
• v.9 no.1
• /
• pp.187-196
• /
• 2024

In this study, we propose a DNN (Deep Neural Network)-based sign language translation model that can significantly reduce training time by compressing sign language coordinates. We compared and analyzed the accuracy and training time of the model with and without sign language coordinate compression. The results of using the proposed model for sign language translation showed that while the accuracy decreased by approximately 5.9% after compressing the sign language video, the training time was reduced by 56.57%, indicating a substantial gain in training efficiency compared to the loss in translation accuracy.

• KIPS Transactions on Software and Data Engineering
• /
• v.11 no.8
• /
• pp.325-330
• /
• 2022

This paper conducts the collection and experiment of datasets for deep learning research on sign language such as sign language recognition, sign language translation, and sign language segmentation for Korean sign language. There exist difficulties for deep learning research of sign language. First, it is difficult to recognize sign languages since they contain multiple modalities including hand movements, hand directions, and facial expressions. Second, it is the absence of training data to conduct deep learning research. Currently, KETI dataset is the only known dataset for Korean sign language for deep learning. Sign language datasets for deep learning research are classified into two categories: Isolated sign language and Continuous sign language. Although several foreign sign language datasets have been collected over time. they are also insufficient for deep learning research of sign language. Therefore, we attempted to collect a large-scale Korean sign language dataset and evaluate it using a baseline model named TSPNet which has the performance of SOTA in the field of sign language translation. The collected dataset consists of a total of 11,402 image and text. Our experimental result with the baseline model using the dataset shows BLEU-4 score 3.63, which would be used as a basic performance of a baseline model for Korean sign language dataset. We hope that our experience of collecting Korean sign language dataset helps facilitate further research directions on Korean sign language.

• The Transactions of the Korea Information Processing Society
• /
• v.13 no.3
• /
• pp.140-147
• /
• 2024

In the Republic of Korea, people with hearing impairments are the second-largest demographic within the registered disability community, following those with physical disabilities. Despite this demographic significance, research on sign language translation technology is limited due to several reasons including the limited market size and the lack of adequately annotated datasets. Despite the difficulties, a few researchers continue to improve the performacne of sign language translation technologies by employing the recent advance of deep learning, for example, the transformer architecture, as the transformer-based models have demonstrated noteworthy performance in tasks such as action recognition and video classification. This study focuses on enhancing the recognition performance of sign language translation by combining transformers with 3D-CNN. Through experimental evaluations using the PHOENIX-Wether-2014T dataset [1], we show that the proposed model exhibits comparable performance to existing models in terms of Floating Point Operations Per Second (FLOPs).

• Annual Conference of KIPS
• /
• 2023.11a
• /
• pp.575-576
• /
• 2023

한국 수어는 소리로 말을 배울 수 없어서 사용하는 '보이는 언어'이고 한국수화언어를 줄인 말이다. 한국어나 영어와 같이 독립된 언어로 한국어와는 문법 체계가 다른 대한민국 농인의 고유한 언어이다. 하지만, 한국 사회에서는 수어를 일상어로 사용하는 농인이 수어만으로 다른 사람과 대화하거나 서비스 등을 이용하기에는 쉽지 않은 구조이다. 이에 본 논문에서는 택시라는 상황을 가정해 택시 안에서 학습된 모델이 농인의 수어를 인식하고 택시 기사에게 해당 의미를 전달하는 시스템을 제안한다. 제안 시스템을 통해 택시 기사는 농인(수어사용자)에게 응답할 수 있다. 본 논문에서는 한국수어 번역기 웹서비스를 설계 및 구현하여 실제 환경에서의 활용 가능성을 검증한다.

• The Journal of the Korea Contents Association
• /
• v.21 no.12
• /
• pp.505-516
• /
• 2021

The purpose of this study is to develop sign language interpreting content in the medical setting that facilitates Deaf people's access to medical services in situations where professional and accurate medical sign language interpreting is insufficient. To achieve the purpose of this study, we conducted a literature review, individual interviews for Deaf people, on-site requirement surveys of sign language interpreters and sign language experts, and medical and sign language expert consultations. Based on this, we developed sign language interpreting content such as main care contextual scenarios, basic medical terms, and medical term descriptions. Through this study, we developed medical sign language content considering the situation and medical importance of Deaf people to promote expertise in the medial sign language area and developed a responsive website of sign language medical dictionary that effectively and efficiently delivers information to Deaf people and sign language interpreters; we realized the need and importance of sign language translation for Deaf people to be the main bodies.

• Annual Conference of KIPS
• /
• 2022.11a
• /
• pp.684-686
• /
• 2022

다양한 언어로 소통하고 있는 우리는 다른 언어와 교류하기 위해 번역, 통역의 존재가 필수가 되기도 한다. 하지만 음성언어를 사용하지 않는 즉, 손으로 언어를 표현하는 수어를 번역하는 통역의 존재는 아직 실현되지 않았다. 이에 본 논문에서는 MediaPipe와 OpenCV 라이브러리를 이용하여 손의 형태를 인식하고 CNN 알고리즘을 통한 텍스트 데이터화 하여 수어 동작을 학습시켜 이를 번역시켜주는 시스템을 연구한다. 이를 통해 공공기관을 이용함에 불편함을 줄이고, 농인의 의사를 보다 빠르게 파악할 수 있도록 도와주는 번역 시스템 제작하는 것에 목적이 있다.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.22 no.3
• /
• pp.63-68
• /
• 2022

There are 395,789 people with hearing impairment in Korea, according to the 2021 Disability Statistics Annual Report by the Korea Institute for the Development of Disabled Persons. These people are experiencing a lot of inconvenience through hearing impairment, and many studies related to recognition and translation of Korean sign language are being conducted to solve this problem. In sign language recognition and translation research, collecting sign language data has many difficulties because few people use sign language professionally. In addition, most of the existed data is sign language data taken from the front of the speaker. To solve this problem, in this paper, we designed and developed a storage system that can collect sign language data based on multi-view points in real-time, rather than a single point, and store and manage it with high usability.