• Smart Media Journal
• /
• v.12 no.1
• /
• pp.32-46
• /
• 2023

Natural Language Processing (NLP) has grown tremendously in recent years. Typically, bilingual, and multilingual translation models have been deployed widely in machine translation and gained vast attention from the research community. On the contrary, few studies have focused on translating between spoken and sign languages, especially non-English languages. Prior works on Sign Language Translation (SLT) have shown that a mid-level sign gloss representation enhances translation performance. Therefore, this study presents a new large-scale Korean sign language dataset, the Museum-Commentary Korean Sign Gloss (MCKSG) dataset, including 3828 pairs of Korean sentences and their corresponding sign glosses used in Museum-Commentary contexts. In addition, we propose a translation framework based on self-supervised learning, where the pretext task is a text-to-text from a Korean sentence to its back-translation versions, then the pre-trained network will be fine-tuned on the MCKSG dataset. Using self-supervised learning help to overcome the drawback of a shortage of sign language data. Through experimental results, our proposed model outperforms a baseline BERT model by 6.22%.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2022.10a
• /
• pp.19-21
• /
• 2022

Anomaly detection using self-supervised learning typically generates synthetic data to learn to classify normal and abnormal, and uses real abnormal data as test data to measure anomaly detection performance. In a study using this method to generate synthetic data similar to normal data, anomaly detection was carried out by generating synthetic data by cutting and pasting a specific patch from the original image. In this way, the degree of similarity to normal data depends on the number and size of patches, which affects anomaly detection performance. In this paper, synthetic data were generated by varying patch sizes and numbers, and then similarity and analysis with normal data were conducted using a pre-trained model, and anomaly detection performance was measured by learning the model.

• The Journal of Information Systems
• /
• v.32 no.4
• /
• pp.119-136
• /
• 2023

Purpose We introduce a groundbreaking surveillance system explicitly designed to overcome the limitations typically associated with conventional surveillance systems, which often focus primarily on object-centric behavior analysis. Design/methodology/approach The study introduces an innovative approach to risk assessment in surveillance, employing image captioning to generate descriptive captions that effectively encapsulate the interactions among objects, actions, and spatial elements within observed scenes. To support our methodology, we developed a distinctive dataset comprising pairs of [image-caption-danger score] for training purposes. We fine-tuned the BLIP-2 model using this dataset and utilized BERT to decipher the semantic content of the generated captions for assessing risk levels. Findings In a series of experiments conducted with our self-constructed datasets, we illustrate that these datasets offer a wealth of information for risk assessment and display outstanding performance in this area. In comparison to models pre-trained on established datasets, our generated captions thoroughly encompass the necessary object attributes, behaviors, and spatial context crucial for the surveillance system. Additionally, they showcase adaptability to novel sentence structures, ensuring their versatility across a range of contexts.

• Journal of the Korean Society of Safety
• /
• v.38 no.5
• /
• pp.51-57
• /
• 2023

Numerous studies have been conducted to analyze the causal relationships of maritime accidents using natural language processing techniques. However, when multiple causes and effects are associated with a single accident, the effectiveness of extracting these causal relations diminishes. To address this challenge, we compiled a dataset using verdicts from maritime accident cases in this study, analyzed their causal relations, and applied labeling considering the association information of various causes and effects. In addition, to validate the efficacy of our proposed methodology, we fine-tuned the KoELECTRA Korean language model. The results of our validation process demonstrated the ability of our approach to successfully extract multiple causal relationships from maritime accident cases.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.17 no.12
• /
• pp.3416-3435
• /
• 2023

Task offloading in vehicular networks is hot topic in the development of autonomous driving. In these scenarios, due to the role of vehicles and pedestrians, task characteristics are changing constantly. The classical deep learning algorithm always uses a pre-trained neural network to optimize task offloading, which leads to system performance degradation. Therefore, this paper proposes a neural network fine-tuning task offloading algorithm, combining with location prediction for pedestrians and vehicles by the Payne model of fluid dynamics and the car-following model, respectively. After the locations are predicted, characteristics of tasks can be obtained and the neural network will be fine-tuned. Finally, the proposed algorithm continuously predicts task characteristics and fine-tunes a neural network to maintain high system performance and meet low delay requirements. From the simulation results, compared with other algorithms, the proposed algorithm still guarantees a lower task offloading delay, especially when congestion occurs.

• International Journal of Internet, Broadcasting and Communication
• /
• v.16 no.3
• /
• pp.212-220
• /
• 2024

A recently the advancement of society, AI technology has made significant strides, especially in the fields of computer vision and voice recognition. This study introduces a system that leverages these technologies to recognize users through a camera and relay commands within a vehicle based on voice commands. The system uses the YOLO (You Only Look Once) machine learning algorithm, widely used for object and entity recognition, to identify specific users. For voice command recognition, a machine learning model based on spectrogram voice analysis is employed to identify specific commands. This design aims to enhance security and convenience by preventing unauthorized access to vehicles and IoT devices by anyone other than registered users. We converts camera input data into YOLO system inputs to determine if it is a person, Additionally, it collects voice data through a microphone embedded in the device or computer, converting it into time-domain spectrogram data to be used as input for the voice recognition machine learning system. The input camera image data and voice data undergo inference tasks through pre-trained models, enabling the recognition of simple commands within a limited space based on the inference results. This study demonstrates the feasibility of constructing a device management system within a confined space that enhances security and user convenience through a simple real-time system model. Finally our work aims to provide practical solutions in various application fields, such as smart homes and autonomous vehicles.

• Journal of the Korea Society of Computer and Information
• /
• v.26 no.4
• /
• pp.21-28
• /
• 2021

This study proposes a deep learning-based flat foot classification methodology using transfer learning. We used a transfer learning with VGG16 pre-trained model and a data augmentation technique to generate a model with high predictive accuracy from a total of 176 image data consisting of 88 flat feet and 88 normal feet. To evaluate the performance of the proposed model, we performed an experiment comparing the prediction accuracy of the basic CNN-based model and the prediction model derived through the proposed methodology. In the case of the basic CNN model, the training accuracy was 77.27%, the validation accuracy was 61.36%, and the test accuracy was 59.09%. Meanwhile, in the case of our proposed model, the training accuracy was 94.32%, the validation accuracy was 86.36%, and the test accuracy was 84.09%, indicating that the accuracy of our model was significantly higher than that of the basic CNN model.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2021.07a
• /
• pp.391-394
• /
• 2021

최근 딥러닝(Deep Learning)을 활용하여 텍스트로 표현된 단어나 문장의 의미를 파악하기 위한 다양한 연구가 활발하게 수행되고 있다. 하지만, 딥러닝을 통해 특정 도메인에서 사용되는 언어를 이해하기 위해서는 해당 도메인의 충분한 데이터에 대해 오랜 시간 학습이 수행되어야 한다는 어려움이 있다. 이러한 어려움을 극복하고자, 최근에는 방대한 양의 데이터에 대한 학습 결과인 사전 학습 언어 모델(Pre-trained Language Model)을 다른 도메인의 학습에 적용하는 방법이 딥러닝 연구에서 많이 사용되고 있다. 이들 접근법은 사전 학습을 통해 단어의 일반적인 의미를 학습하고, 이후에 단어가 특정 도메인에서 갖는 의미를 파악하기 위해 추가적인 학습을 진행한다. 추가 학습에는 일반적으로 대표적인 사전 학습 언어 모델인 BERT의 MLM(Masked Language Model)이 다시 사용되며, 마스크(Mask) 되지 않은 단어들의 의미로부터 마스크 된 단어의 의미를 추론하는 형태로 학습이 이루어진다. 따라서 사전 학습을 통해 의미가 파악되어 있는 단어들이 마스크 되지 않고, 신조어와 같이 의미가 알려져 있지 않은 단어들이 마스크 되는 비율이 높을수록 단어 의미의 학습이 정확하게 이루어지게 된다. 하지만 기존의 MLM은 무작위로 마스크 대상 단어를 선정하므로, 사전 학습을 통해 의미가 파악된 단어와 사전 학습에 포함되지 않아 의미 파악이 이루어지지 않은 신조어가 별도의 구분 없이 마스크에 포함된다. 따라서 본 연구에서는 사전 학습에 포함되지 않았던 신조어에 대해서만 집중적으로 마스킹(Masking)을 수행하는 방안을 제시한다. 이를 통해 신조어의 의미 학습이 더욱 정확하게 이루어질 수 있고, 궁극적으로 이러한 학습 결과를 활용한 후속 분석의 품질도 향상시킬 수 있을 것으로 기대한다. 영화 정보 제공 사이트인 N사로부터 영화 댓글 12만 건을 수집하여 실험을 수행한 결과, 제안하는 신조어 표적 마스킹(NTM: Newly Coined Words Target Masking)이 기존의 무작위 마스킹에 비해 감성 분석의 정확도 측면에서 우수한 성능을 보임을 확인하였다.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2022.10a
• /
• pp.164-167
• /
• 2022

In table Machine comprehension, the knowledge required for language models or the structural form of tables changes depending on the domain, showing a greater performance degradation compared to text data. In this paper, we propose a pre-learning data construction method and an adversarial learning method through meaningful tabular data selection for constructing a pre-learning table language model robust to these domain changes in table machine reading. In order to detect tabular data sed for decoration of web documents without structural information from the extracted table data, a rule through heuristic was defined to identify head data and select table data was applied. An adversarial learning method between tabular data and infobax data with knowledge information about entities was applied. When the data was refined compared to when it was trained with the existing unrefined data, F1 3.45 and EM 4.14 increased in the KorQuAD table data, and F1 19.38, EM 4.22 compared to when the data was not refined in the Spec table QA data showed increased performance.

• Journal of Intelligence and Information Systems
• /
• v.24 no.3
• /
• pp.1-19
• /
• 2018

Large amount of data is now available for research and business sectors to extract knowledge from it. This data can be in the form of unstructured data such as audio, text, and image data and can be analyzed by deep learning methodology. Deep learning is now widely used for various estimation, classification, and prediction problems. Especially, fashion business adopts deep learning techniques for apparel recognition, apparel search and retrieval engine, and automatic product recommendation. The core model of these applications is the image classification using Convolutional Neural Networks (CNN). CNN is made up of neurons which learn parameters such as weights while inputs come through and reach outputs. CNN has layer structure which is best suited for image classification as it is comprised of convolutional layer for generating feature maps, pooling layer for reducing the dimensionality of feature maps, and fully-connected layer for classifying the extracted features. However, most of the classification models have been trained using online product image, which is taken under controlled situation such as apparel image itself or professional model wearing apparel. This image may not be an effective way to train the classification model considering the situation when one might want to classify street fashion image or walking image, which is taken in uncontrolled situation and involves people's movement and unexpected pose. Therefore, we propose to train the model with runway apparel image dataset which captures mobility. This will allow the classification model to be trained with far more variable data and enhance the adaptation with diverse query image. To achieve both convergence and generalization of the model, we apply Transfer Learning on our training network. As Transfer Learning in CNN is composed of pre-training and fine-tuning stages, we divide the training step into two. First, we pre-train our architecture with large-scale dataset, ImageNet dataset, which consists of 1.2 million images with 1000 categories including animals, plants, activities, materials, instrumentations, scenes, and foods. We use GoogLeNet for our main architecture as it has achieved great accuracy with efficiency in ImageNet Large Scale Visual Recognition Challenge (ILSVRC). Second, we fine-tune the network with our own runway image dataset. For the runway image dataset, we could not find any previously and publicly made dataset, so we collect the dataset from Google Image Search attaining 2426 images of 32 major fashion brands including Anna Molinari, Balenciaga, Balmain, Brioni, Burberry, Celine, Chanel, Chloe, Christian Dior, Cividini, Dolce and Gabbana, Emilio Pucci, Ermenegildo, Fendi, Giuliana Teso, Gucci, Issey Miyake, Kenzo, Leonard, Louis Vuitton, Marc Jacobs, Marni, Max Mara, Missoni, Moschino, Ralph Lauren, Roberto Cavalli, Sonia Rykiel, Stella McCartney, Valentino, Versace, and Yve Saint Laurent. We perform 10-folded experiments to consider the random generation of training data, and our proposed model has achieved accuracy of 67.2% on final test. Our research suggests several advantages over previous related studies as to our best knowledge, there haven't been any previous studies which trained the network for apparel image classification based on runway image dataset. We suggest the idea of training model with image capturing all the possible postures, which is denoted as mobility, by using our own runway apparel image dataset. Moreover, by applying Transfer Learning and using checkpoint and parameters provided by Tensorflow Slim, we could save time spent on training the classification model as taking 6 minutes per experiment to train the classifier. This model can be used in many business applications where the query image can be runway image, product image, or street fashion image. To be specific, runway query image can be used for mobile application service during fashion week to facilitate brand search, street style query image can be classified during fashion editorial task to classify and label the brand or style, and website query image can be processed by e-commerce multi-complex service providing item information or recommending similar item.