• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.17 no.3
• /
• pp.881-895
• /
• 2023

Modern image inpainting techniques based on deep learning have achieved remarkable performance, and more and more people are working on repairing more complex and larger missing areas, although this is still challenging, especially for facial image inpainting. For a face image with a huge missing area, there are very few valid pixels available; however, people have an ability to imagine the complete picture in their mind according to their subjective will. It is important to simulate this capability while maintaining the identity features of the face as much as possible. To achieve this goal, we propose a three-stage network model, which we refer to as the identity and structure feature refinement network (ISFRNet). ISFRNet is based on 1) a pre-trained pSp-styleGAN model that generates an extremely realistic face image with rich structural features; 2) a shallow structured network with a small receptive field; and 3) a modified U-net with two encoders and a decoder, which has a large receptive field. We choose structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), L1 Loss and learned perceptual image patch similarity (LPIPS) to evaluate our model. When the missing region is 20%-40%, the above four metric scores of our model are 28.12, 0.942, 0.015 and 0.090, respectively. When the lost area is between 40% and 60%, the metric scores are 23.31, 0.840, 0.053 and 0.177, respectively. Our inpainting network not only guarantees excellent face identity feature recovery but also exhibits state-of-the-art performance compared to other multi-stage refinement models.

• Journal of Drive and Control
• /
• v.21 no.2
• /
• pp.53-64
• /
• 2024

Global population growth has resulted in an increased demand for food production. Simultaneously, aging rural communities have led to a decrease in the workforce, thereby increasing the demand for automation in agriculture. Drones are particularly useful for unmanned pest control fields. However, the current method of uniform spraying leads to environmental damage due to overuse of pesticides and drift by wind. To address this issue, it is necessary to enhance spraying performance through precise performance evaluation. Therefore, as a foundational study aimed at optimizing drone-based pest control technologies, this research evaluated water-sensitive paper (WSP) via density map estimation using convolutional neural networks (CNN) with a encoder-decoder structure. To achieve more accurate estimation, this study implemented multi-task learning, incorporating an additional classifier for image segmentation alongside the density map estimation classifier. The proposed model in this study resulted in a R-squared (R²) of 0.976 for coverage area in the evaluation data set, demonstrating satisfactory performance in evaluating WSP at various density levels. Further research is needed to improve the accuracy of spray result estimations and develop a real-time assessment technology in the field.

• Journal of the Korea Institute of Information Security & Cryptology
• /
• v.34 no.3
• /
• pp.379-392
• /
• 2024

With the rapid increase in data, saving storage space and improving the efficiency of data transmission have become critical issues, making the research on the efficiency of data compression technologies increasingly important. Lossless algorithms can precisely restore original data but have limited compression ratios, whereas lossy algorithms provide higher compression rates at the expense of some data loss. There has been active research in data compression using deep learning-based algorithms, especially the autoencoder model. This study proposes a new side-channel analysis data compressor utilizing autoencoders. This compressor achieves higher compression rates than Deflate while maintaining the characteristics of side-channel data. The encoder, using locally connected layers, effectively preserves the temporal characteristics of side-channel data, and the decoder maintains fast decompression times with a multi-layer perceptron. Through correlation power analysis, the proposed compressor has been proven to compress data without losing the characteristics of side-channel data.

• The Journal of Korean Institute of Electromagnetic Engineering and Science
• /
• v.21 no.6
• /
• pp.651-659
• /
• 2010

In this paper, a new six-port direct conversion receiver for high-speed multi-band multi-mode wireless communication system such as software defined radio(SDR) is proposed. The designed receiver is composed of two CMOS four-port BPSK receivers and a dual-band one-stage polyphase filter for quadrature LO signal generation. The four-port BPSK receiver, implemented in 0.18 ${\mu}m$ CMOS technology for the first time in microwave-band, is composed of two active combiners, an active balun, two power detector, and an analog decoder. The proposed polyphase filter adopt type-I architecture, one-stage for reduction of the local oscillator power loss, and LC resonance structure instead of using capacitor for dual-band operation. In order to extent the operation RF bandwidth of the proposed six-port receiver, we include I/Q phase and amplitude calibration scheme in the six-port junction and the power detector. The calibration range of the phase and amplitude mismatch in the proposed calibration scheme is 8 degree and 14 dB, respectively. The validity of the designed six-port receiver is successfully demonstrated by modulating M-QAM, and M-PSK signal with 40 Msps in the two-band of 900 MHz and 2.4 GHz.

• Smart Media Journal
• /
• v.11 no.10
• /
• pp.65-75
• /
• 2022

Deep learning is used as a creative tool that could overcome the limitations of existing analysis models and generate various types of results such as text, image, and music. In this paper, we propose a method necessary to preprocess audio data using the Niko's MIDI Pack sound source file as a data set and to generate music using Bi-LSTM. Based on the generated root note, the hidden layers are composed of multi-layers to create a new note suitable for the musical composition, and an attention mechanism is applied to the output gate of the decoder to apply the weight of the factors that affect the data input from the encoder. Setting variables such as loss function and optimization method are applied as parameters for improving the LSTM model. The proposed model is a multi-channel Bi-LSTM with attention that applies notes pitch generated from separating treble clef and bass clef, length of notes, rests, length of rests, and chords to improve the efficiency and prediction of MIDI deep learning process. The results of the learning generate a sound that matches the development of music scale distinct from noise, and we are aiming to contribute to generating a harmonistic stable music.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.47 no.3
• /
• pp.87-95
• /
• 2010

Portable multi-media products which can service the highest audio-quality by using lossless audio codec has been released and the international lossless codecs, MPEG-4 audio lossless coding(ALS) and MPEG-4 scalable lossless coding(SLS), were standardized by MPEG in 2006. The simple profile of MPEG-4 ALS, it supports up to stereo, was defined by MPEG in 2009. The lossless audio codec should have low-complexity in stereo to be widely used in portable multi-media products. But the previous researches of MPEG-4 ALS have focused on an improvement of compression ratio, a complexity reduction in multi-channels coding, and a selection of linear prediction coefficients(LPCs) order. In this paper, the complexity and compression ratio of MPEG-4 ALS encoder is analyzed in simple profile of MPEG-4 ALS, the method to reduce a complexity of MPEG-4 ALS encoder is proposed. Based on an analysis of complexity of MPEG-4 ALS encoder, the complexity of short-term prediction filter of MPEG-4 ALS encoder is reduced by using the low-complexity filter that is proposed in previous research to reduce the complexity of MPEG-4 ALS decoder. Also, we propose a joint coding decision method, it reduces the complexity and keeps the compression ratio of MPEG-4 ALS encoder. In proposed method, the operation of joint coding is decided based on the relation between cross-correlation of residual and compression ratio of joint coding. The performance of MPEG-4 ALS encoder that has the method and low-complexity filter is evaluated by using the MPEG-4 ALS conformance test file and normal music files. The complexity of MPEG-4 ALS encoder is reduced by about 24% by comparing with MPEG-4 ALS reference encoder, while the compression ratio by the proposed method is comparable to MPEG-4 ALS reference encoder.

• Journal of the Institute of Electronics Engineers of Korea SP
• /
• v.47 no.6
• /
• pp.48-56
• /
• 2010

Recently, we need to dramatically speed up real-time video encoding and decoding on mobile devices because complexity of video CODEC is significantly increasing along with the demand for multimedia service of high-quality and high-definition videos by users. A variety of research is conducted for parallelism of video processing using newly developed multi-core platforms. In this paper, we propose a method of parallelism based on slice partition of video compression CODEC. We propose a novel concept of a parallel slice for parallelism and propose a new coding order to be adequate to the parallel slice which keeps high coding efficiency. To minimize synchronization time of multiple parallel slices, we also propose a synchronization method to determinate whether the parallel slice could be independently decoded or not. Experimental results shows that we achieved 27.5% (40.7%) speed-up by parallelism with bit-rate increase of 3.4% (2.7%) for CIF sequences (720p sequences) by implementing the proposed algorithm on the H.264/AVC.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.48 no.1
• /
• pp.116-126
• /
• 2011

In this paper, we present a multi-media processor which can decode multiple-format video standards. The designed processor is evaluated with optimized MPEG-2, MPEG-4, and AVS (Audio video standard). There are two approaches for developing of real-time video decoders. First, hardware-based system is much superior to a processor-based one in execution time. However, it takes long time to implement and modify hardware systems. On the contrary, the software-based video codecs can be easily implemented and flexible, however, their performance is not so good for real-time applications. In this paper, in order to exploit benefits related to two approaches, we designed a processor called ASIP(Application specific instruction-set processor) for video decoding. In our work, we extracted eight common modules from various video decoders, and added several multimedia instructions to the processor. The developed processor for video decoders is evaluated with the Synopsys platform simulator and a FPGA board. In our experiment, we can achieve about 37% time saving in total decoding time.

• Journal of KIISE:Computer Systems and Theory
• /
• v.33 no.12
• /
• pp.867-879
• /
• 2006

This paper concerns fast and time accurate HW/SW cosimulation for MPSoC(Multi-Processor System-on-chip) architecture where multiple software and/or hardware components exist. It is becoming more and more common to use MPSoC architecture to design complex embedded systems. In cosimulation of such architecture, as the number of the component simulators participating in the cosimulation increases, the time synchronization overhead among simulators increases, thereby resulting in low overall cosimulation performance. Although SystemC cosimulation frameworks show high cosimulation performance, it is in inverse proportion to the number of simulators. In this paper, we extend the novel technique, called virtual synchronization, which boosts cosimulation speed by reducing time synchronization overhead: (1) SystemC simulation is supported seamlessly in the virtual synchronization framework without requiring the modification on SystemC kernel (2) Parallel execution of component simulators with virtual synchronization is supported. We compared the performance and accuracy of the proposed parallel SystemC cosimulation framework with MaxSim, a well-known commercial SystemC cosimulation framework, and the proposed one showed 11 times faster performance for H.263 decoder example, while the accuracy was maintained below 5%.