• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.15 no.4
• /
• pp.1464-1485
• /
• 2021

Emotional style of multimedia art works are abstract content information. This study aims to explore emotional style transfer method and find the possible way of matching music with appropriate images in respect to emotional style. DCNNs (Deep Convolutional Neural Networks) can capture style and provide emotional style transfer iterative solution for affective image generation. Here, we learn the image emotion features via DCNNs and map the affective style on the other images. We set image emotion feature as the style target in this style transfer problem, and held experiments to handle affective image generation of eight emotion categories, including dignified, dreaming, sad, vigorous, soothing, exciting, joyous, and graceful. A user study was conducted to test the synesthesia emotional image style transfer result with ground truth user perception triggered by the music-image pairs' stimuli. The transferred affective image result for music-image emotional synesthesia perception was proved effective according to user study result.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2019.05a
• /
• pp.475-478
• /
• 2019

Although the work of neural style transfer has shown successful applications in transferring the style of a certain type of artistic painting, it is less effective in transferring Oriental paintings. In this paper, we explore three methods which are effective in transferring Oriental paintings. Then, we take a typical network from each method to carry on the experiment, in view of three different methods to Oriental paintings style transfer effect has carried on the discussion.

• Journal of Broadcast Engineering
• /
• v.27 no.1
• /
• pp.104-123
• /
• 2022

Style transfer based on neural network provides very high quality results by reflecting the high level structural characteristics of images, and thereby has recently attracted great attention. This paper deals with the problem of resolution limitation due to GPU memory in performing such neural style transfer. We can expect that the gradient operation for style transfer based on partial image, with the aid of the fixed size of receptive field, can produce the same result as the gradient operation using the entire image. Based on this idea, each component of the style transfer loss function is analyzed in this paper to obtain the necessary conditions for partitioning and padding, and to identify, among the information required for gradient calculation, the one that depends on the entire input. By structuring such information for using it as auxiliary constant input for partition-based gradient calculation, this paper develops a recursive algorithm for super high-resolution image style transfer. Since the proposed method performs style transfer by partitioning input image into the size that a GPU can handle, it can perform style transfer without the limit of the input image resolution accompanied by the GPU memory size. With the aid of such super high-resolution support, the proposed method can provide a unique style characteristics of detailed area which can only be appreciated in super high-resolution style transfer.

• Journal of the Korea Institute of Military Science and Technology
• /
• v.24 no.4
• /
• pp.382-391
• /
• 2021

Unsupervised domain adaptation often gives impressive solutions to handle domain shift of data. Most of current approaches assume that unlabeled target data to train is abundant. This assumption is not always true in practices. To tackle this issue, we propose a general solution to solve the domain gap minimization problem without any target data. Our method consists of two regularization steps. The first step is a pixel regularization by arbitrary style transfer. Recently, some methods bring style transfer algorithms to domain adaptation and domain generalization process. They use style transfer algorithms to remove texture bias in source domain data. We also use style transfer algorithms for removing texture bias, but our method depends on neither domain adaptation nor domain generalization paradigm. The second regularization step is a feature regularization by feature alignment. Adding a feature alignment loss term to the model loss, the model learns domain invariant representation more efficiently. We evaluate our regularization methods from several experiments both on small dataset and large dataset. From the experiments, we show that our model can learn domain invariant representation as much as unsupervised domain adaptation methods.

• Journal of Information Processing Systems
• /
• v.20 no.1
• /
• pp.53-66
• /
• 2024

With the continuous advancement of computer technology, deep learning models have emerged as innovative tools in shaping various aspects of architectural design. Recognizing the distinctive perspective of children, which differs significantly from that of adults, this paper contends that conventional standards may not always be the most suitable approach in designing urban structures tailored for children. The primary objective of this study is to leverage neural style networks within the design process, specifically adopting the artistic viewpoint found in children's illustrations. By combining the aesthetic paradigm of urban architecture with inspiration drawn from children's aesthetic preferences, the aim is to unearth more creative and subversive aesthetics that challenge traditional norms. The selected context for exploration is the landmark buildings in Qingdao City, Shandong Province, China. Employing the neural style network, the study uses architectural elements of the chosen buildings as content images while preserving their inherent characteristics. The process involves artistic stylization inspired by classic children's illustrations and images from children's picture books. Acting as a conduit for deep learning technology, the research delves into the prospect of seamlessly integrating architectural design styles with the imaginative world of children's illustrations. The outcomes aim to provide fresh perspectives and effective support for the artistic design of contemporary urban buildings.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2023.05a
• /
• pp.724-725
• /
• 2023

본 논문에서는 Unity3D 엔진을 이용하여 과거와 현재를 연결하는 내용에 기반한 미디어아트 콘텐츠를 구현하였다. Super Zoom Neural Filters를 사용하여 이미지의 화질을 개선하였으며 Neural Style Transfer를 사용하여 현대 건물을 민속화 스타일로 변환시켰다. 이 작품은 과거와 현재가 연결되는 미디어아트 작품으로서의 의의를 찾을 수 있으며 민속화 활용 미디어아트의 사례로 평가받을 수 있다고 생각한다.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2022.06a
• /
• pp.1339-1342
• /
• 2022

예술적 스타일 전이는 예술 작품이 지닌 특징을 다른 이미지에 적용하는 이미지 처리의 오랜 화두 중 하나로, 최근에는 StyleGAN 과 같이 미리 학습된 GAN(생성적 적대 신경망)을 통해 제한된 데이터로도 고해상도의 예술적 초상화를 생성하도록 학습하는 연구가 다양한 방면에서 성과를 내고 있다. 본 논문에서는 2 가지 경로의 StyleGAN과 Facial Destylization 을 통해 고해상도의 예시 기반 스타일 전이를 달성한 DualStyleGAN 연구에 대해 소개하고, 기존 연구에서 사용된 Facial Destylization 방법이 지닌 한계점을 분석한 뒤, 이를 개선한 새로운 방법, Re-Destyle을 제안한다. 새로운 Re-Destyle 방법으로 Facial Destylization 을 적용할 경우 학습 시간을 기존 연구의 방법보다 20 배 이상 개선할 수 있으며 그 결과 1000 개 이하의 적은 데이터와 1~2 시간의 추가 학습만으로도 원하는 타겟 초상화 스타일에 대해 1024×1024 수준의 고해상도의 예시 기반 초상화 스타일 전이 및 이미지 생성 모델을 학습할 수 있다.

• KIPS Transactions on Software and Data Engineering
• /
• v.11 no.11
• /
• pp.465-472
• /
• 2022

In this paper, the style synthesis network is trained to generate style-synthesized video through the style synthesis through training Stylegan and the video synthesis network for video synthesis. In order to improve the point that the gaze or expression does not transfer stably, 3D face restoration technology is applied to control important features such as the pose, gaze, and expression of the head using 3D face information. In addition, by training the discriminators for the dynamics, mouth shape, image, and gaze of the Head2head network, it is possible to create a stable style synthesis video that maintains more probabilities and consistency. Using the FaceForensic dataset and the MetFace dataset, it was confirmed that the performance was increased by converting one video into another video while maintaining the consistent movement of the target face, and generating natural data through video synthesis using 3D face information from the source video's face.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.17 no.2
• /
• pp.291-298
• /
• 2022

A generative adversarial network (GAN) is a network in which two internal neural networks (generative network and discriminant network) learn while competing with each other. The generator creates an image close to reality, and the delimiter is programmed to better discriminate the image of the constructor. This technology is being used in various ways to create, transform, and restore the entire image X into another image Y. This paper describes a method that can be forged into another object naturally, after extracting only a partial image from the original image. First, a new image is created through the previously trained DCGAN model, after extracting only a partial image from the original image. The original image goes through a process of naturally combining with, after re-styling it to match the texture and size of the original image using the overall style transfer technique. Through this study, the user can naturally add/transform the desired object image to a specific part of the original image, so it can be used as another field of application for creating fake images.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2021.07a
• /
• pp.667-669
• /
• 2021

본 논문에서는 벡터 기반 데이터 증강 기법(Data augmentation)을 제안하여 학습 데이터를 구축한 뒤, 이를 합성곱 신경망(Convolutional Neural Networks, CNN)으로 실제 균열과 가까운 패턴을 표현할 수 있는 프레임워크를 제안한다. 건축물의 균열은 인명 피해를 가져오는 건물 붕괴와 낙하 사고를 비롯한 큰 사고의 원인이다. 이를 인공지능으로 해결하기 위해서는 대량의 데이터 확보가 필수적이다. 하지만, 실제 균열 이미지는 복잡한 패턴을 가지고 있을 뿐만 아니라, 위험한 상황에 노출되기 때문에 대량의 데이터를 확보하기 어렵다. 이러한 데이터베이스 구축의 문제점은 인위적으로 특정 부분에 변형을 주어 데이터양을 늘리는 탄성왜곡(Elastic distortion) 기법으로 해결할 수 있지만, 본 논문에서는 이보다 향상된 균열 패턴 결과를 CNN을 활용하여 보여준다. 탄성왜곡 기법보다 CNN을 이용했을 때, 실제 균열 패턴과 유사하게 추출된 결과를 얻을 수 있었고, 일반적으로 사용되는 픽셀 기반 데이터가 아닌 벡터 기반으로 데이터 증강을 설계함으로써 균열의 변화량 측면에서 우수함을 보였다. 본 논문에서는 적은 개수의 균열 데이터를 입력으로 사용했음에도 불구하고 균열의 방향 및 패턴을 다양하게 생성하여 쉽게 균열 데이터베이스를 구축할 수 있었다. 이는 장기적으로 구조물의 안정성 평가에 이바지하여 안전사고에 대한 불안감에서 벗어나 더욱 안전하고 쾌적한 주거 환경을 조성할 것으로 기대된다.