• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.41 no.3
• /
• pp.55-65
• /
• 2004

When a neural network is used to solve a given problem it is necessary to match the complexity of the network to that of the problem because the complexity of the network significantly affects its learning capability and generalization performance. Thus, it is desirable to have an algorithm that can find appropriate network structures in a self-organizing way. This paper proposes algorithms which automatically organize feed forward multi-layer neural networks with sigmoid hidden neurons for given problems. Using both constructive procedures and pruning procedures, the proposed algorithms try to find the near optimal network, which is compact and shows good generalization performance. The performances of the proposed algorithms are tested on four function regression problems. The results demonstrate that our algorithms successfully generate near-optimal networks in comparison with the previous method and the neural networks of fixed topology.

• Electronics and Telecommunications Trends
• /
• v.38 no.5
• /
• pp.12-22
• /
• 2023

The development of neural networks is evolving towards the adoption of transformer structures with attention modules. Hence, active research focused on extending the concept of lightweight neural network algorithms and hardware acceleration is being conducted for the transition from conventional convolutional neural networks to transformer-based networks. We present a survey of state-of-the-art research on lightweight neural network algorithms and hardware architectures to reduce memory usage and accelerate both inference and training. To describe the corresponding trends, we review recent studies on token pruning, quantization, and architecture tuning for the vision transformer. In addition, we present a hardware architecture that incorporates lightweight algorithms into artificial intelligence processors to accelerate processing.

• Journal of the Korean Operations Research and Management Science Society
• /
• v.27 no.1
• /
• pp.33-51
• /
• 2002

A fully connected Artificial Neural Network (ANN) contains many connections. Compared to the pruned ANN with fewer connections, the fully connected ANN takes longer time to produce solutions end may not provide appropriate solutions to new unseen date. Therefore, by reducing the sloe of ANN, we can overcome the overfitting problem and increase the computing speed. In this research, we reduced the size of ANN by destroying the connections. In other words, we investigated the performance change of the reduced ANN by systematically destroying the connections. Then we found the acceptable level of connection-destruction on which the resulting ANN Performs as well as the original fully connected ANN. In the previous researches on the sloe reduction of ANN, the reduced ANN had to be retrained every time some connections were eliminated. Therefore, It tool lolly time to obtain the reduced ANN. In this research, however, we provide the acceptable level of connection-destruction according to the size of the fully connected ANN. Therefore, by applying the acceptable level of connection-destruction to the fully connected ANN without any retraining, the reduced ANN can be obtained efficiently.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 2003.09a
• /
• pp.620-623
• /
• 2003

The TAM (Topographic Attentive Mapping) network is a biologically-motivated neural network. Fuzzy rules are acquired from the TAM network by the pruning algorithm. In this paper we formulate a new input layer using Gabor function for TAU network to realize receptive field of human visual cortex.

• Journal of Korea Society of Industrial Information Systems
• /
• v.22 no.5
• /
• pp.17-24
• /
• 2017

Recently Deep-learning, Technologies using Large or Deep Artificial Neural Networks, have Shown Remarkable Performance, and the Increasing Size of the Network Contributes to its Performance Improvement. However, the Increase in the Size of the Neural Network Leads to an Increase in the Calculation Amount, which Causes Problems Such as Circuit Complexity, Price, Heat Generation, and Real-time Restriction. In This Paper, We Propose and Test a Method to Reduce the Number of Network Connections by Effectively Pruning the Redundancy in the Connection and Showing the Difference between the Performance and the Desired Range of the Original Neural Network. In Particular, we Proposed a Simple Method to Improve the Performance by Re-learning and to Guarantee the Desired Performance by Allocating the Error Rate per Layer in Order to Consider the Difference of each Layer. Experiments have been Performed on a Typical Neural Network Structure such as FCN (full connection network) and CNN (convolution neural network) Structure and Confirmed that the Performance Similar to that of the Original Neural Network can be Obtained by Only about 1/10 Connection.

• Journal of Positioning, Navigation, and Timing
• /
• v.8 no.4
• /
• pp.193-200
• /
• 2019

The auto-encoder network which is a good candidate to handle the modeling of the signal strength attenuation is designed for denoising and compensating the distortion of the received data. It provides a non-linear mapping function by iteratively learning the encoder and the decoder. The encoder is the non-linear mapping function, and the decoder demands accurate data reconstruction from the representation generated by the encoder. In addition, the adaptive network width which supports the automatic generation of new hidden nodes and pruning of inconsequential nodes is also implemented in the proposed algorithm for increasing the efficiency of the algorithm. Simulation results show that the proposed method can improve the neural network training surface to achieve the highest possible accuracy of the signal modeling compared with the conventional modeling method.

• Journal of KIISE:Software and Applications
• /
• v.30 no.7_8
• /
• pp.649-658
• /
• 2003

We propose a hierarchical architecture of ART2 Network for performance improvement and fast pattern classification model using fitness selection. This hierarchical network creates coarse clusters as first ART2 network layer by unsupervised learning, then creates fine clusters of the each first layer as second network layer by supervised learning. First, it compares input pattern with each clusters of first layer and select candidate clusters by fitness measure. We design a optimized fitness function for pruning clusters by measuring relative distance ratio between a input pattern and clusters. This makes it possible to improve speed and accuracy. Next, it compares input pattern with each clusters connected with selected clusters and finds winner cluster. Finally it classifies the pattern by a label of the winner cluster. Results of our experiments show that the proposed method is more accurate and fast than other approaches.

• Journal of Broadcast Engineering
• /
• v.25 no.2
• /
• pp.200-207
• /
• 2020

Recently, deep neural networks such as CNN are showing excellent performance in various fields such as image classification, object recognition, visual quality enhancement, etc. However, as the model size and computational complexity of deep learning models for most applications increases, it is hard to apply neural networks to IoT and mobile environments. Therefore, neural network compression algorithms for reducing the model size while keeping the performance have been being studied. In this paper, we apply few compression methods to CNN models and evaluate their performances in the embedded environment. For evaluate the performance, the classification performance and inference time of the original CNN models and the compressed CNN models on the image inputted by the camera are evaluated in the embedded board equipped with QCS605, which is a customized AI chip. In this paper, a few CNN models of MobileNetV2, ResNet50, and VGG-16 are compressed by applying the methods of pruning and matrix decomposition. The experimental results show that the compressed models give not only the model size reduction of 1.3~11.2 times at a classification performance loss of less than 2% compared to the original model, but also the inference time reduction of 1.2~2.21 times, and the memory reduction of 1.2~3.8 times in the embedded board.

• Proceedings of the Korean Institute of Intelligent Systems Conference
• /
• 2003.09a
• /
• pp.415-418
• /
• 2003

Fuzzy modelling has the approximation property far the given input-output relationship. Especially, Takagi-Sugeno fuzzy models are widely used because they show very good performance in the nonlinear function approximation problem. But generally there is not the systematic method incorporating the human expert's knowledge or experience in fuzzy rules and it is not easy to End the membership function of fuzzy rule to minimize the output error as well. The ANFIS (Adaptive Network-based Fuzzy Inference Systems) is one of the neural network based fuzzy modelling methods that can be used with various type of fuzzy rules. But in this model, it is the problem to End the optimum number of fuzzy rules in fuzzy model. In this paper, a new fuzzy modelling method based on the ANFIS and pruning techniques with the measure named impact factor is proposed and the performance of proposed method is evaluated with several simulation results.

• Journal of Intelligence and Information Systems
• /
• v.4 no.1
• /
• pp.79-87
• /
• 1998

정확한 태풍진로 예측은 동아시아 최대의 자연재해인 태풍의 피해를 최소화하는데 필수적이다. 기상역학에 기초를 둔 수치모델과 회귀분석등의 통계적 접근법이 사용되어왔다. 본 논문에서는 비선형 신경망모델인 다층퍼셉트론을 제안한다. 즉, 태풍진로예측을 이동경로, 속도, 기압 등의 변수로 이루어진 시계열의 예측으로 본다. 1945년부터 1989년까지 한반도에 접근한 태풍 데이터를 이용하여 제안된 신경망을 학습한 후, 94, 95년도에 접근한 태풍의 진로를 예측하였다. 신경망의 예측성능은 수치모델의 성능보다 조금 우수하거나 비슷하였다. 신경망의 성능은 충분히 더 향상될 수 있는 여지가 있다. 또한, 고가의 슈퍼컴퓨터로 여러 시간 계산을 해야하는 수치모델에 비하여 PC상에서 수초만에 계산을 할 수 있는 신경망 모델은 비용 면에서도 장점이 있다.