• ETRI Journal
• /
• v.44 no.5
• /
• pp.794-804
• /
• 2022

Transformer models have had a great impact on natural language processing (NLP) in recent years by realizing outstanding and efficient contextualized language models. Recent studies have used transformer-based language models for various NLP tasks, including Persian named entity recognition (NER). However, in complex tasks, for example, NER, it is difficult to determine which contextualized embedding will produce the best representation for the tasks. Considering the lack of comparative studies to investigate the use of different contextualized pretrained models with sequence modeling classifiers, we conducted a comparative study about using different classifiers and embedding models. In this paper, we use different transformer-based language models tuned with different classifiers, and we evaluate these models on the Persian NER task. We perform a comparative analysis to assess the impact of text representation and text classification methods on Persian NER performance. We train and evaluate the models on three different Persian NER datasets, that is, MoNa, Peyma, and Arman. Experimental results demonstrate that XLM-R with a linear layer and conditional random field (CRF) layer exhibited the best performance. This model achieved phrase-based F-measures of 70.04, 86.37, and 79.25 and word-based F scores of 78, 84.02, and 89.73 on the MoNa, Peyma, and Arman datasets, respectively. These results represent state-of-the-art performance on the Persian NER task.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.18 no.1
• /
• pp.19-24
• /
• 2023

In traditional fish farming way, the workers have to observe all of the pools every time and every day to feed at the right timing. This method causes tremendous stress on workers and wastes time. To solve this problem, we implemented an automatic detection system for feeding time using deep neural network. The detection system consists of two steps: classification of the presence or absence of feed and checking DO (Dissolved Oxygen) of the pool. For the classification, the pretrained ResNet18 model and transfer learning with custom dataset are used. DO is obtained from the DO sensor in the pool through HTTP in real time. For better accuracy, the next step, checking DO proceeds when the result of the classification is absence of feed several times in a row. DO is checked if it is higher than a DO reference value that is set by the workers. These actions are performed automatically in the UI programs developed with LabVIEW.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2021.11a
• /
• pp.749-752
• /
• 2021

오늘날 인공지능(Artificial Intelligence, AI) 분야에서 가장 구현하기 어려운 분야 중 하나는 추론이다. 근래 추론 분야에서 영상과 언어가 결합한 다중 모드(Multi-modal) 환경에서 영상 기반의 질의 응답(Visual Question Answering, VQA) 과업에 대한 AI 모델이 발표됐다. 얼마 지나지 않아 VQA 모델의 성능을 개선한 GVQA(Grounded Visual Question Answering) 모델도 발표됐다. 하지만 아직 GVQA 모델도 완벽한 성능을 내진 못한다. 본 논문에서는 GVQA 모델의 성능 개선을 위해 VCC(Visual Concept Classifier) 모델을 ViT-G(Vision Transformer-Giant)/14로 변경하고, ACP(Answer Cluster Predictor) 모델을 GPT(Generative Pretrained Transformer)-3으로 변경한다. 이와 같은 방법들은 성능을 개선하는 데 큰 도움이 될 수 있다고 사료된다.

• Annual Conference on Human and Language Technology
• /
• 2023.10a
• /
• pp.711-715
• /
• 2023

본 연구는 대부분의 언어 모델이 사용하고 있는 서브워드 토큰화 과정을 거치지 않고, 바이트 단위의 인코딩을 그대로 다룰 수 있는 토큰-프리 사전학습 언어모델에 대한 것이다. 토큰-프리 언어모델은 명시적인 미등록어 토큰이 존재하지 않고, 전 처리 과정이 단순하며 다양한 언어 및 표현 체계에 대응할 수 있는 장점이 있다. 하지만 관련 연구가 미흡, 서브워드 모델에 대비해 학습이 어렵고 낮은 성능이 보고되어 왔다. 본 연구에서는 한국어를 중심으로 토큰-프리 언어 이해-생성 모델을 사전 학습 후, 서브워드 기반 모델과 비교하여 가능성을 살펴본다. 또한, 토큰 프리 언어모델에서 지적되는 과도한 연산량을 감소시킬 수 있는 그래디언트 기반 서브워드 토크나이저를 적용, 처리 속도를 학습 2.7배, 추론 1.46배 개선하였다.

• Korean Journal of Radiology
• /
• v.24 no.10
• /
• pp.952-959
• /
• 2023

Large language models (LLMs) such as ChatGPT have garnered considerable interest for their potential to aid non-native English-speaking researchers. These models can function as personal, round-the-clock English tutors, akin to how Prometheus in Greek mythology bestowed fire upon humans for their advancement. LLMs can be particularly helpful for non-native researchers in writing the Introduction and Discussion sections of manuscripts, where they often encounter challenges. However, using LLMs to generate text for research manuscripts entails concerns such as hallucination, plagiarism, and privacy issues; to mitigate these risks, authors should verify the accuracy of generated content, employ text similarity detectors, and avoid inputting sensitive information into their prompts. Consequently, it may be more prudent to utilize LLMs for editing and refining text rather than generating large portions of text. Journal policies concerning the use of LLMs vary, but transparency in disclosing artificial intelligence tool usage is emphasized. This paper aims to summarize how LLMs can lower the barrier to academic writing in English, enabling researchers to concentrate on domain-specific research, provided they are used responsibly and cautiously.

• Imaging Science in Dentistry
• /
• v.50 no.2
• /
• pp.169-174
• /
• 2020

Purpose: Periodontal disease causes tooth loss and is associated with cardiovascular diseases, diabetes, and rheumatoid arthritis. The present study proposes using a deep learning-based object detection method to identify periodontally compromised teeth on digital panoramic radiographs. A faster regional convolutional neural network (faster R-CNN) which is a state-of-the-art deep detection network, was adapted from the natural image domain using a small annotated clinical data- set. Materials and Methods: In total, 100 digital panoramic radiographs of periodontally compromised patients were retrospectively collected from our hospital's information system and augmented. The periodontally compromised teeth found in each image were annotated by experts in periodontology to obtain the ground truth. The Keras library, which is written in Python, was used to train and test the model on a single NVidia 1080Ti GPU. The faster R-CNN model used a pretrained ResNet architecture. Results: The average precision rate of 0.81 demonstrated that there was a significant region of overlap between the predicted regions and the ground truth. The average recall rate of 0.80 showed that the periodontally compromised teeth regions generated by the detection method excluded healthiest teeth areas. In addition, the model achieved a sensitivity of 0.84, a specificity of 0.88 and an F-measure of 0.81. Conclusion: The faster R-CNN trained on a limited amount of labeled imaging data performed satisfactorily in detecting periodontally compromised teeth. The application of a faster R-CNN to assist in the detection of periodontally compromised teeth may reduce diagnostic effort by saving assessment time and allowing automated screening documentation.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.33 no.6
• /
• pp.1087-1098
• /
• 2023

The rapid advancement of artificial intelligence (AI) technology has led to its proactive utilization across various fields. However, this widespread adoption of AI-based systems has raised concerns about the increasing threat of attacks on these systems. In particular, deep neural networks, commonly used in deep learning, have been found vulnerable to adversarial attacks that intentionally manipulate input data to induce model errors. In this study, we propose a method to protect image classification models from visually imperceptible One-Pixel attacks, where only a single pixel is altered in an image. The proposed defense technique utilizes an autoencoder model to remove potential threat elements from input images before forwarding them to the classification model. Experimental results, using the CIFAR-10 dataset, demonstrate that the autoencoder-based defense approach significantly improves the robustness of pretrained image classification models against One-Pixel attacks, with an average defense rate enhancement of 81.2%, all without the need for modifications to the existing models.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.26 no.5
• /
• pp.1259-1267
• /
• 2016

With the growing use of fingerprint authentication systems in recent years, the fake fingerprint detection is becoming more and more important. This paper mainly proposes a method for fake fingerprint detection based on CNN, it will visualize the distinctive part of detected fingerprint which provides a deeper insight in CNN model. After the preprocessing part using fingerprint segmentation, the pretrained CNN model is used for detecting the liveness detection. Not only a liveness detection but also feature analysis about the live fingerprint and fake fingerprint are provided after classifying which materials are used for making the fake fingerprint. Our system is evaluated on three databases in LivDet2013, which compromise almost 6500 live fingerprint images and 6000 fake fingerprint images in total. The proposed method achieves 3.1% ACE value about the liveness detection and achieves 79.58% accuracy on LiveDet2013.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.25 no.1
• /
• pp.44-49
• /
• 2021

Recently, due to COVID-19, studies have been popularly worked to apply neural network to mask wearing automatic detection system. For applying neural networks, the 1-stage detection or 2-stage detection methods are used, and if data are not sufficiently collected, the pretrained neural network models are studied by applying fine-tuning techniques. In this paper, the system is consisted of 2-stage detection method that contain MTCNN model for face recognition and ResNet model for mask detection. The mask detector was experimented by applying five ResNet models to improve accuracy and fps in various environments. Training data used 17,217 images that collected using web crawler, and for inference, we used 1,913 images and two one-minute videos respectively. The experiment showed a high accuracy of 96.39% for images and 92.98% for video, and the speed of inference for video was 10.78fps.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.25 no.11
• /
• pp.1505-1511
• /
• 2021

Pretrained models with high performance in various tasks within natural language processing have the advantage of learning the linguistic patterns of sentences using large corpus during the training, allowing each token in the input sentence to be represented with appropriate feature vectors. One of the methods of constructing a corpus required for a pre-trained model training is a collection method using web crawler. However, sentences that exist on web may contain unnecessary words in some or all of the sentences because they have various patterns. In this paper, we propose a dataset construction method for filtering sentences containing unnecessary words using neural network models for corpus collected from the web. As a result, we construct a dataset containing a total of 2,330 sentences. We also evaluated the performance of neural network models on the constructed dataset, and the BERT model showed the highest performance with an accuracy of 93.75%.