• Journal of Information Processing Systems
• /
• v.14 no.1
• /
• pp.176-190
• /
• 2018

This paper presents a novel convolutional neural network based multi-feature fusion learning method for non-rigid 3D model retrieval, which can investigate the useful discriminative information of the heat kernel signature (HKS) descriptor and the wave kernel signature (WKS) descriptor. At first, we compute the 2D shape distributions of the two kinds of descriptors to represent the 3D model and use them as the input to the networks. Then we construct two convolutional neural networks for the HKS distribution and the WKS distribution separately, and use the multi-feature fusion layer to connect them. The fusion layer not only can exploit more discriminative characteristics of the two descriptors, but also can complement the correlated information between the two kinds of descriptors. Furthermore, to further improve the performance of the description ability, the cross-connected layer is built to combine the low-level features with high-level features. Extensive experiments have validated the effectiveness of the designed multi-feature fusion learning method.

• IEMEK Journal of Embedded Systems and Applications
• /
• v.17 no.4
• /
• pp.217-228
• /
• 2022

Currently, there are increasing demands for applying deep neural networks (DNNs) in the embedded domain such as classification and object detection. The DNN processing in embedded domain often requires custom hardware such as NPU for acceleration due to the constraints in power, performance, and area. Processing DNN models requires a large amount of data, and its seamless transfer to NPU is crucial for performance. In this paper, we developed a cycle-accurate NPU simulator to evaluate diverse NPU microarchitectures. In addition, we propose a novel technique for reducing the number of memory accesses when processing convolutional layers in convolutional neural networks (CNNs) on the NPU. The main idea is to reuse data with memory interleaving, which recycles the overlapping data between previous and current input windows. Data memory interleaving makes it possible to quickly read consecutive data in unaligned locations. We implemented the proposed technique to the cycle-accurate NPU simulator and measured the performance with LeNet-5, VGGNet-16, and ResNet-50. The experiment shows up to 2.08x speedup in processing one convolutional layer, compared to the baseline.

• Journal of Information Processing Systems
• /
• v.19 no.6
• /
• pp.803-816
• /
• 2023

Explainable artificial intelligence is a method that explains how a complex model (e.g., a deep neural network) yields its output from a given input. Recently, graph-type data have been widely used in various fields, and diverse graph neural networks (GNNs) have been developed for graph-type data. However, methods to explain the behavior of GNNs have not been studied much, and only a limited understanding of GNNs is currently available. Therefore, in this paper, we propose an explanation method for node classification using graph convolutional networks (GCNs), which is a representative type of GNN. The proposed method finds out which features of each node have the greatest influence on the classification of that node using GCN. The proposed method identifies influential features by backtracking the layers of the GCN from the output layer to the input layer using the gradients. The experimental results on both synthetic and real datasets demonstrate that the proposed explanation method accurately identifies the features of each node that have the greatest influence on its classification.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.18 no.2
• /
• pp.311-326
• /
• 2024

The rapid development of neural network technology promotes the neural network model driven by big data to overcome the texture effect of complex objects. Due to the limitations in complex scenes, it is necessary to establish custom template matching and apply it to the research of many fields of computational vision technology. The dependence on high-quality small label sample database data is not very strong, and the machine learning system of deep feature connection to complete the task of texture effect inference and speculation is relatively poor. The style transfer algorithm based on neural network collects and preserves the data of patterns, extracts and modernizes their features. Through the algorithm model, it is easier to present the texture color of patterns and display them digitally. In this paper, according to the texture effect reasoning of custom template matching, the 3D visualization of the target is transformed into a 3D model. The high similarity between the scene to be inferred and the user-defined template is calculated by the user-defined template of the multi-dimensional external feature label. The convolutional neural network is adopted to optimize the external area of the object to improve the sampling quality and computational performance of the sample pyramid structure. The results indicate that the proposed algorithm can accurately capture the significant target, achieve more ablation noise, and improve the visualization results. The proposed deep convolutional neural network optimization algorithm has good rapidity, data accuracy and robustness. The proposed algorithm can adapt to the calculation of more task scenes, display the redundant vision-related information of image conversion, enhance the powerful computing power, and further improve the computational efficiency and accuracy of convolutional networks, which has a high research significance for the study of image information conversion.

• KSCE Journal of Civil and Environmental Engineering Research
• /
• v.44 no.3
• /
• pp.385-396
• /
• 2024

To enhance the accuracy of sinkhole detection using GPR, this study derived a convolutional neural network that can optimally extract sinkhole characteristics from GPR B-scan grayscale images. The pre-trained convolutional neural network is evaluated to be more than twice as effective as the vanilla convolutional neural network. In pre-trained convolutional neural networks, fast feature extraction is found to cause less overfitting than feature extraction. It is analyzed that the top-1 verification accuracy and computation time are different depending on the type of architecture and simulation conditions. Among the pre-trained convolutional neural networks, InceptionV3 are evaluated as most robust for sinkhole detection in GPR B-scan grayscale images. When considering both top-1 verification accuracy and architecture efficiency index, VGG19 and VGG16 are analyzed to have high efficiency as the backbone for extracting sinkhole feature from GPR B-scan grayscale images. MobileNetV3-Large backbone is found to be suitable when mounted on GPR equipment to extract sinkhole feature in real time.

• Journal of Korea Multimedia Society
• /
• v.25 no.11
• /
• pp.1643-1652
• /
• 2022

Most of the existing machine reading research has used Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN) algorithms as networks. Among them, RNN was slow in training, and Question Answering Network (QANet) was announced to improve training speed. QANet is a model composed of CNN and self-attention. CNN extracts semantic and syntactic information well from the local corpus, but there is a limit to extracting the corresponding information from the global corpus. Graph Convolutional Networks (GCN) extracts semantic and syntactic information relatively well from the global corpus. In this paper, to take advantage of this strength of GCN, we propose I-QANet, which changed the CNN of QANet to GCN. The proposed model performed 1.2 times faster than the baseline in the Stanford Question Answering Dataset (SQuAD) dataset and showed 0.2% higher performance in Exact Match (EM) and 0.7% higher in F1. Furthermore, in the Korean Question Answering Dataset (KorQuAD) dataset consisting only of Korean, the learning time was 1.1 times faster than the baseline, and the EM and F1 performance were also 0.9% and 0.7% higher, respectively.

• Journal of the Semiconductor & Display Technology
• /
• v.21 no.4
• /
• pp.156-162
• /
• 2022

Image compression plays an important role in encoding and improving various forms of images in the digital era. Recent researches have focused on the principle of deep learning as one of the most exciting machine learning methods to show that it is good scheme to analyze, classify and compress images. Various neural networks are able to adapt for image compressions, such as deep neural networks, artificial neural networks, recurrent neural networks and convolution neural networks. In this review paper, we discussed how to apply the rule of deep learning to obtain better image compression with high accuracy, low loss-ness and high visibility of the image. For those results in performance, deep learning methods are required on justified manner with distinct analysis.

• The Korean Journal of Applied Statistics
• /
• v.33 no.5
• /
• pp.603-614
• /
• 2020

This study focuses on the prediction and factors of apartment prices in Seoul using a convolutional neural networks (CNN) model that has shown excellent performance as a predictive model of image data. To do this, we consider natural environmental factors, infrastructure factors, and social economic factors of the apartments as input variables of the CNN model. The natural environmental factors include rivers, green areas, and altitudes of apartments. The infrastructure factors have bus stops, subway stations, commercial districts, schools, and the social economic factors are the number of jobs and criminal rates, etc. We predict apartment prices and interpret the factors for the prices by converting the values of these input variables to play the same role as pixel values of image channels for the input layer in the CNN model. In addition, the CNN model used in this study takes into account the spatial characteristics of each apartment by describing the natural environmental and infrastructure factors variables as binary images centered on each apartment in each input layer.

• The Journal of Korean Institute of Information Technology
• /
• v.16 no.11
• /
• pp.85-95
• /
• 2018

With the popularity of smartphones, most peoples have been using mobile cameras to capture photographs. However, due to insufficient amount of lights in a low lighting condition, unwanted noises can be generated during image acquisition. To remove the noise, a method of using deep convolutional neural networks is introduced. However, this method still lacks the ability to describe textures and edges, even though it has made significant progress in terms of visual quality performance. Therefore, in this paper, the HOG (Histogram of Oriented Gradients) images that contain information about edge orientations are used. More specifically, a method of learning deep convolutional neural networks is proposed by stacking noise and HOG images into an input tensor. Experiment results confirm that the proposed method not only can obtain excellent result in visual quality evaluations, compared to conventional methods, but also enable textures and edges to be improved visually.

• Journal of Multimedia Information System
• /
• v.4 no.4
• /
• pp.225-232
• /
• 2017

Understanding actions in videos is an important task. It helps in finding the anomalies present in videos such as fights. Detection of fights becomes more crucial when it comes to sports. This paper focuses on finding fight scenes in Hockey sport videos using blur & radon transform and convolutional neural networks (CNNs). First, the local motion within the video frames has been extracted using blur information. Next, fast fourier and radon transform have been applied on the local motion. The video frames with fight scene have been identified using transfer learning with the help of pre-trained deep learning model VGG-Net. Finally, a comparison of the methodology has been performed using feed forward neural networks. Accuracies of 56.00% and 75.00% have been achieved using feed forward neural network and VGG16-Net, respectively.