• Journal of the Korea Society of Computer and Information
• /
• v.26 no.4
• /
• pp.11-19
• /
• 2021

An abnormal object refers to a person, an object, or a mechanical device that performs abnormal and unusual behavior and needs observation or supervision. In order to detect this through artificial intelligence algorithm without continuous human intervention, a method of observing the specificity of temporal features using optical flow technique is widely used. In this study, an abnormal situation is identified by learning an algorithm that translates an input image frame to an optical flow image using a Generative Adversarial Network (GAN). In particular, we propose a technique that improves the pre-processing process to exclude unnecessary outliers and the post-processing process to increase the accuracy of identification in the test dataset after learning to improve the performance of the model's abnormal behavior identification. UCSD Pedestrian and UMN Unusual Crowd Activity were used as training datasets to detect abnormal behavior. For the proposed method, the frame-level AUC 0.9450 and EER 0.1317 were shown in the UCSD Ped2 dataset, which shows performance improvement compared to the models in the previous studies.

• The Journal of the Acoustical Society of Korea
• /
• v.38 no.6
• /
• pp.670-677
• /
• 2019

As the presence of background noise in acoustic signal degrades the performance of speech or acoustic event recognition, it is still challenging to extract noise-robust acoustic features from noisy signal. In this paper, we propose a combined structure of Wasserstein Generative Adversarial Network (WGAN) and MultiTask AutoEncoder (MTAE) as deep learning architecture that integrates the strength of MTAE and WGAN respectively such that it estimates not only noise but also speech features from noisy acoustic source. The proposed MTAE-WGAN structure is used to estimate speech signal and the residual noise by employing a gradient penalty and a weight initialization method for Leaky Rectified Linear Unit (LReLU) and Parametric ReLU (PReLU). The proposed MTAE-WGAN structure with the adopted gradient penalty loss function enhances the speech features and subsequently achieve substantial Phoneme Error Rate (PER) improvements over the stand-alone Deep Denoising Autoencoder (DDAE), MTAE, Redundant Convolutional Encoder-Decoder (R-CED) and Recurrent MTAE (RMTAE) models for robust speech recognition.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.26 no.9
• /
• pp.1305-1311
• /
• 2022

With the recent rapid development of mobile terminals and personal computers and the advent of neural network technology, real-time face swapping using images has become possible. In particular, the cycle generative adversarial network made it possible to replace faces using uncorrelated image data. In this paper, we propose an input data processing scheme that can improve the quality of face swapping with less training data and time. The proposed scheme can improve the image quality while preserving facial structure and expression information by combining facial landmarks extracted through a pre-trained neural network with major information that affects the structure and expression of the face. Using the blind/referenceless image spatial quality evaluator (BRISQUE) score, which is one of the AI-based non-reference quality metrics, we quantitatively analyze the performance of the proposed scheme and compare it to the conventional schemes. According to the numerical results, the proposed scheme obtained BRISQUE scores improved by about 4.6% to 14.6%, compared to the conventional schemes.

• Journal of Internet Computing and Services
• /
• v.21 no.4
• /
• pp.9-16
• /
• 2020

In the Image processing and computer vision, the problem of reconstructing from one image to another or generating a new image has been steadily drawing attention as hardware advances. However, the problem of computer-generated images also continues to emerge when viewed with human eyes because it is not natural. Due to the recent active research in deep learning, image generating and improvement problem using it are also actively being studied, and among them, the network called Generative Adversarial Network(GAN) is doing well in the image generating. Various models of GAN have been presented since the proposed GAN, allowing for the generation of more natural images compared to the results of research in the image generating. Among them, pix2pix is a conditional GAN model, which is a general-purpose network that shows good performance in various datasets. pix2pix is based on U-Net, but there are many networks that show better performance among U-Net based networks. Therefore, in this study, images are generated by applying various networks to U-Net of pix2pix, and the results are compared and evaluated. The images generated through each network confirm that the pix2pix model with Attention, R2, and Attention-R2 networks shows better performance than the existing pix2pix model using U-Net, and check the limitations of the most powerful network. It is suggested as a future study.

• Journal of Ocean Engineering and Technology
• /
• v.36 no.1
• /
• pp.32-40
• /
• 2022

Underwater optical images face various limitations that degrade the image quality compared with optical images taken in our atmosphere. Attenuation according to the wavelength of light and reflection by very small floating objects cause low contrast, blurry clarity, and color degradation in underwater images. We constructed an image data of the Korean sea and enhanced it by learning the characteristics of underwater images using the deep learning techniques of CycleGAN (cycle-consistent adversarial network), UGAN (underwater GAN), FUnIE-GAN (fast underwater image enhancement GAN). In addition, the underwater optical image was enhanced using the image processing technique of Image Fusion. For a quantitative performance comparison, UIQM (underwater image quality measure), which evaluates the performance of the enhancement in terms of colorfulness, sharpness, and contrast, and UCIQE (underwater color image quality evaluation), which evaluates the performance in terms of chroma, luminance, and saturation were calculated. For 100 underwater images taken in Korean seas, the average UIQMs of CycleGAN, UGAN, and FUnIE-GAN were 3.91, 3.42, and 2.66, respectively, and the average UCIQEs were measured to be 29.9, 26.77, and 22.88, respectively. The average UIQM and UCIQE of Image Fusion were 3.63 and 23.59, respectively. CycleGAN and UGAN qualitatively and quantitatively improved the image quality in various underwater environments, and FUnIE-GAN had performance differences depending on the underwater environment. Image Fusion showed good performance in terms of color correction and sharpness enhancement. It is expected that this method can be used for monitoring underwater works and the autonomous operation of unmanned vehicles by improving the visibility of underwater situations more accurately.

• Imaging Science in Dentistry
• /
• v.52 no.4
• /
• pp.393-398
• /
• 2022

Purpose: This study proposed a generative adversarial network (GAN) model for T2-weighted image (WI) synthesis from proton density (PD)-WI in a temporomandibular joint(TMJ) magnetic resonance imaging (MRI) protocol. Materials and Methods: From January to November 2019, MRI scans for TMJ were reviewed and 308 imaging sets were collected. For training, 277 pairs of PD- and T2-WI sagittal TMJ images were used. Transfer learning of the pix2pix GAN model was utilized to generate T2-WI from PD-WI. Model performance was evaluated with the structural similarity index map (SSIM) and peak signal-to-noise ratio (PSNR) indices for 31 predicted T2-WI (pT2). The disc position was clinically diagnosed as anterior disc displacement with or without reduction, and joint effusion as present or absent. The true T2-WI-based diagnosis was regarded as the gold standard, to which pT2-based diagnoses were compared using Cohen's ĸ coefficient. Results: The mean SSIM and PSNR values were 0.4781(±0.0522) and 21.30(±1.51) dB, respectively. The pT2 protocol showed almost perfect agreement(ĸ=0.81) with the gold standard for disc position. The number of discordant cases was higher for normal disc position (17%) than for anterior displacement with reduction (2%) or without reduction (10%). The effusion diagnosis also showed almost perfect agreement(ĸ=0.88), with higher concordance for the presence (85%) than for the absence (77%) of effusion. Conclusion: The application of pT2 images for a TMJ MRI protocol useful for diagnosis, although the image quality of pT2 was not fully satisfactory. Further research is expected to enhance pT2 quality.

• Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
• /
• v.40 no.4
• /
• pp.359-364
• /
• 2022

The change detection in remote sensing results in errors due to the camera's optical factors, seasonal factors, and land cover characteristics. The inclination of the building in the image was simulated according to the camera angle using the Cycle Generative Adversarial Network method, and the simulated image was used to contribute to the improvement of change detection accuracy. Based on CycleGAN, the inclination of the building was similarly simulated to the building in the other image based on the image of one of the two periods, and the error of the original image and the inclination of the building was compared and analyzed. The experimental data were taken at different times at different angles, and Kompsat-3A high-resolution satellite images including urban areas with dense buildings were used. As a result of the experiment, the number of incorrect detection pixels per building in the two images for the building area in the image was shown to be reduced by approximately 7 times from 12,632 in the original image and 1,730 in the CycleGAN-based simulation image. Therefore, it was confirmed that the proposed method can reduce detection errors due to the inclination of the building.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2019.10a
• /
• pp.976-979
• /
• 2019

사회가 발전함에 따라 사람들의 의류 소비 패턴 매장 매출보다 모바일 쇼핑몰 매출이 늘고 있다. 스마트 뱅킹, 쇼핑몰 애플리케이션 등 모바일 서비스가 일상생활로 스며들면서 모바일로 의류를 구매하는 것은 쉬워졌다. 하지만, 온라인이라는 특성상 옷을 택배로 받고, 입어야 옷이 어울리는 지 아닌 지를 판단할 수 있다는 고질적인 문제점이 있다. 이러한 문제점은 반품 또는 교환으로 이루어지고 이는 쇼핑몰과 소비자 모두에게 굉장히 낭비되는 비용이다. 본 논문에서는 사람의 사진에 옷을 입힌 사진을 제공함으로써, 사람이 옷을 실제로 입지 않더라도 그 때의 fit 을 제공하고자 한다. 이때, 단순한 합성이 아니라, 딥러닝 중 GAN(Generative Adversarial Network)를 사용해 기존 기술의 문제점을 해결하고자 한다.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2019.07a
• /
• pp.255-256
• /
• 2019

한글 폰트를 만드는 데는 자음+모음 조합으로 약 11,500자 정도의 글자가 필요하다. 디자이너가 글자 하나씩 전부 디자인 하는 것도 굉장한 부담요소이고, 한글폰트를 제작하는데 있어 3개월 이상의 소요 기간과 3000만 원 이상의 비용부담 또한 무시 못 할 요소이다. 게다가 카피라이트 폰트에 대한 저작권 문제 또한 골칫거리다. 그래서 이를 최소한으로 하고자 딥 러닝의 방식중 하나인 GAN(생성적 적대 신경망)을 통해서 디자이너가 399자만 작성하고 나머지는 컴퓨터가 디자이너의 폰트 디자인을 인식하고 자동으로 만들어 주는 프로그램을 고안하였다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2023.05a
• /
• pp.479-481
• /
• 2023

Generative Adversarial Networks (GANs) have facilitated substantial improvement in single-image super-resolution (SR) by enabling the generation of photo-realistic images. However, the high memory requirements of GAN-based SRs (mainly generators) lead to reduced performance and increased energy consumption, making it difficult to implement them onto resource-constricted devices. In this study, we propose an efficient and compressed architecture for the SR-GAN (generator) model using the model compression technique Knowledge Distillation. Our approach involves the transmission of knowledge from a heavy network to a lightweight one, which reduces the storage requirement of the model by 58% with also an increase in their performance. Experimental results on various benchmarks indicate that our proposed compressed model enhances performance with an increase in PSNR, SSIM, and image quality respectively for x4 super-resolution tasks.