• Journal of Information Processing Systems
• /
• v.18 no.1
• /
• pp.12-25
• /
• 2022

In recent years, deep convolutional neural networks have made significant progress in the research of single image super-resolution. However, it is difficult to be applied in practical computing terminals or embedded devices due to a large number of parameters and computational effort. To balance these problems, we propose CSRNet, a lightweight neural network based on channel split residual learning structure, to reconstruct highresolution images from low-resolution images. Lightweight refers to designing a neural network with fewer parameters and a simplified structure for lower memory consumption and faster inference speed. At the same time, it is ensured that the performance of recovering high-resolution images is not degraded. In CSRNet, we reduce the parameters and computation by channel split residual learning. Simultaneously, we propose a double-upsampling network structure to improve the performance of the lightweight super-resolution network and make it easy to train. Finally, we propose a new evaluation metric for the lightweight approaches named 100_FPS. Experiments show that our proposed CSRNet not only speeds up the inference of the neural network and reduces memory consumption, but also performs well on single image super-resolution.

• Korean Journal of Remote Sensing
• /
• v.36 no.5_2
• /
• pp.961-973
• /
• 2020

In this manuscript, a new pansharpening model based on Convolutional Neural Network (CNN) was developed. Dilated convolution, which is one of the representative convolution technologies in CNN, was applied to the model by making it deep and complex to improve the performance of the deep learning architecture. Based on the dilated convolution, the residual network is used to enhance the efficiency of training process. In addition, we consider the spatial correlation coefficient in the loss function with traditional L1 norm. We experimented with Dilated Residual Networks (DRNet), which is applied to the structure using only a panchromatic (PAN) image and using both a PAN and multispectral (MS) image. In the experiments using KOMPSAT-3A, DRNet using both a PAN and MS image tended to overfit the spectral characteristics, and DRNet using only a PAN image showed a spatial resolution improvement over existing CNN-based models.

• Journal of Korea Multimedia Society
• /
• v.23 no.12
• /
• pp.1540-1551
• /
• 2020

Aiming at the problem that the existing human behavior recognition algorithm cannot fully utilize the multi-level spatio-temporal information of the network, a human behavior recognition algorithm based on a dense three-dimensional residual network is proposed. First, the proposed algorithm uses a dense block of three-dimensional residuals as the basic module of the network. The module extracts the hierarchical features of human behavior through densely connected convolutional layers; Secondly, the local feature aggregation adaptive method is used to learn the local dense features of human behavior; Then, the residual connection module is applied to promote the flow of feature information and reduced the difficulty of training; Finally, the multi-layer local feature extraction of the network is realized by cascading multiple three-dimensional residual dense blocks, and use the global feature aggregation adaptive method to learn the features of all network layers to realize human behavior recognition. A large number of experimental results on benchmark datasets KTH show that the recognition rate (top-l accuracy) of the proposed algorithm reaches 93.52%. Compared with the three-dimensional convolutional neural network (C3D) algorithm, it has improved by 3.93 percentage points. The proposed algorithm framework has good robustness and transfer learning ability, and can effectively handle a variety of video behavior recognition tasks.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2019.10a
• /
• pp.1137-1140
• /
• 2019

가상현실(Virtual Reality, VR), 증강현실(Augmented Reality, AR), 혼합현실(Mixed Reality, MR) 분야에 유용한 인간 컴퓨터 인터페이스 기술은 필수적이다. 특히 휴먼 손동작 인식 기술은 직관적인 상호작용을 가능하게 하여, 다양한 분야에서 편리한 컨트롤러로 사용할 수 있다. 본 연구에서는 뎁스 영상 기반의 1 인칭 시점 손동작 인식을 위하여 손동작 데이터베이스 생성 시스템을 구축하여, 손동작 인식기 학습에 필요한 1 인칭(Egocentric View Point) 데이터베이스를 촬영하여 제작한다. 그리고 모바일 Head Mounted Device(HMD) VR 을 위한 뎁스 영상 기반 1 인칭 시점 손동작 인식(Hand Pose Recognition, HPR) 딥러닝 Deep Residual Network 를 구현한다. 최종적으로, 안드로이드 모바일 디바이스에 학습된 Residual Network Regressor 를 이식하고 모바일 VR 에 실시간 손동작 인식 시스템을 구동하여, 모바일 VR 상 실시간 3D 손동작 인식을 가상 물체와의 상호작용을 통하여 확인 한다.

• Journal of the Korea Society of Computer and Information
• /
• v.25 no.6
• /
• pp.35-47
• /
• 2020

Box-office prediction is important to movie stakeholders. It is necessary to accurately predict box-office and select important variables. In this paper, we propose a multivariate time series classification and important variable selection method to improve accuracy of predicting the box-office. As a research method, we collected daily data from KOBIS and NAVER for South Korean movies, selected important variables using Random Forest and predicted multivariate time series using Deep Learning. Based on the Korean screen quota system, Deep Learning was used to compare the accuracy of box-office predictions on the 73^rd day from movie release with the important variables and entire variables, and the results was tested whether they are statistically significant. As a Deep Learning model, Multi-Layer Perceptron, Fully Convolutional Neural Networks, and Residual Network were used. Among the Deep Learning models, the model using important variables and Residual Network had the highest prediction accuracy at 93%.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.16 no.9
• /
• pp.2882-2903
• /
• 2022

Image super-resolution (SR) processing is of great value in the fields of digital image processing, intelligent security, film and television production and so on. This paper proposed a densely connected deep learning network based on cascade architecture, which can be used to solve the problem of super-resolution in the field of image quality enhancement. We proposed a more efficient residual scaling dense block (RSDB) and the multi-channel cascade architecture to realize more efficient feature reuse. Also we proposed a hybrid loss function based on L₁ error and L_∞ error to achieve better L_∞ error performance. The experimental results show that the overall performance of the network is effectively improved on cascade architecture and residual scaling. Compared with the residual dense net (RDN), the PSNR / SSIM of the new method is improved by 2.24% / 1.44% respectively, and the L_∞ performance is improved by 3.64%. It shows that the cascade connection and residual scaling method can effectively realize feature reuse, improving the residual convergence speed and learning efficiency of our network. The L_∞ performance is improved by 11.09% with only a minimal loses of 1.14% / 0.60% on PSNR / SSIM performance after adopting the new loss function. That is to say, the L_∞ performance can be improved greatly on the new loss function with a minor loss of PSNR / SSIM performance, which is of great value in L_∞ error sensitive tasks.

• International Journal of Computer Science & Network Security
• /
• v.22 no.10
• /
• pp.237-245
• /
• 2022

The prominence of IoTs (Internet of Things) and exponential advancement of computer networks has resulted in massive essential applications. Recognizing various cyber-attacks or anomalies in networks and establishing effective intrusion recognition systems are becoming increasingly vital to current security. MLTs (Machine Learning Techniques) can be developed for such data-driven intelligent recognition systems. Researchers have employed a TFDNNs (Tensor Flow Deep Neural Networks) and DCNNs (Deep Convolution Neural Networks) to recognize pirated software and malwares efficiently. However, tuning the amount of neurons in multiple layers with activation functions leads to learning error rates, degrading classifier's reliability. HTFDNNs ( Hybrid tensor flow DNNs) and MRNs (Modified Residual Networks) or Resnet CNNs were presented to recognize software piracy and malwares. This study proposes HTFDNNs to identify stolen software starting with plagiarized source codes. This work uses Tokens and weights for filtering noises while focusing on token's for identifying source code thefts. DLTs (Deep learning techniques) are then used to detect plagiarized sources. Data from Google Code Jam is used for finding software piracy. MRNs visualize colour images for identifying harms in networks using IoTs. Malware samples of Maling dataset is used for tests in this work.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2018.11a
• /
• pp.59-61
• /
• 2018

Highly compressed images typically not only have low resolution, but are also affected by compression artifacts. Performing image super-resolution (SR) directly on highly compressed image would simultaneously magnify the blocking artifacts. In this paper, a SR method based on deep learning is proposed. The method is an end-to-end trainable deep convolutional neural network which performs SR on compressed images so as to reduce compression artifacts and improve image resolution. The proposed network is divided into compression artifacts removal (CAR) part and SR reconstruction part, and the network is trained by three-step training method to optimize training procedure. Experiments on JPEG compressed images with quality factors of 10, 20, and 30 demonstrate the effectiveness of the proposed method on commonly used test images and image sets.

• ETRI Journal
• /
• v.43 no.3
• /
• pp.538-548
• /
• 2021

This paper proposes an approach to improve the performance of no-reference video quality assessment for sports videos with dynamic motion scenes using an efficient spatiotemporal model. In the proposed method, we divide the video sequences into video blocks and apply a 3D shearlet transform that can efficiently extract primary spatiotemporal features to capture dynamic natural motion scene statistics from the incoming video blocks. The concatenation of a deep residual bidirectional gated recurrent neural network and logistic regression is used to learn the spatiotemporal correlation more robustly and predict the perceptual quality score. In addition, conditional video block-wise constraints are incorporated into the objective function to improve quality estimation performance for the entire video. The experimental results show that the proposed method extracts spatiotemporal motion information more effectively and predicts the video quality with higher accuracy than the conventional no-reference video quality assessment methods.

• Journal of Information Processing Systems
• /
• v.19 no.6
• /
• pp.756-766
• /
• 2023

To effectively track the illegal use of digital images and maintain the security of digital image communication on the Internet, this paper proposes a reversible multipurpose image watermarking algorithm based on a deep residual network (ResNet) and perceptual hashing (also called MWR). The algorithm first combines perceptual image hashing to generate a digital fingerprint that depends on the user's identity information and image characteristics. Then it embeds the removable visible watermark and digital fingerprint in two different regions of the orthogonal separation of the image. The embedding strength of the digital fingerprint is computed using ResNet. Because of the embedding of the removable visible watermark, the conflict between the copyright notice and the user's browsing is balanced. Moreover, image authentication and traitor tracking are realized through digital fingerprint insertion. The experiments show that the scheme has good visual transparency and watermark visibility. The use of chaotic mapping in the visible watermark insertion process enhances the security of the multipurpose watermark scheme, and unauthorized users without correct keys cannot effectively remove the visible watermark.