• Title/Summary/Keyword: network pruning

Search Result 82, Processing Time 0.027 seconds

A Pruning Algorithm for Network Structure Optimization in the Forecasting Climate System Using Neural Network (신경망을 이용한 기상예측시스템에서 망구조 최적화를 위한 Pruning 알고리즘)

  • Lee, Kee-Jun;Kang, Myung-A;Jung, Chai-Yeoung
    • The Transactions of the Korea Information Processing Society
    • /
    • v.7 no.2
    • /
    • pp.385-391
    • /
    • 2000
  • Recently, neural network research for forecasting the consecutive controlling rules of the future is being progressed, using the series data which are different from the traditional statistical analysis methods. In this paper, we suggest the pruning algorithm for the fast and exact weather forecast that excludes the hidden layer of the early optional designed nenral network. There are perform the weather forecast experiments using the 22080 kinds of weather data gathered from 1987 to 1996 for proving the efficiency of this suggested algorithm. Through the experiments, the early optional composed $26{\times}50{\times}1$ nenral network became the most suitable $26{\times}2{\times}1$ structure through the pruning algorithm suggested, in the optimum neural network $26{\times}2{\times}1$, in the case of the error temperature ${\pm}0.5^{\circ}C$, the average was 33.55%, in the case of ${\pm}1^{\circ}C$, the average was 61.57%, they showed more superior than the average 29.31% and 54.47% of the optional designed structure, also. we can reduce the calculation frequency more than maximum 25 times as compared with the optional sturcture neural network in the calculation frequencies.

  • PDF

A hybrid deep neural network compression approach enabling edge intelligence for data anomaly detection in smart structural health monitoring systems

  • Tarutal Ghosh Mondal;Jau-Yu Chou;Yuguang Fu;Jianxiao Mao
    • Smart Structures and Systems
    • /
    • v.32 no.3
    • /
    • pp.179-193
    • /
    • 2023
  • This study explores an alternative to the existing centralized process for data anomaly detection in modern Internet of Things (IoT)-based structural health monitoring (SHM) systems. An edge intelligence framework is proposed for the early detection and classification of various data anomalies facilitating quality enhancement of acquired data before transmitting to a central system. State-of-the-art deep neural network pruning techniques are investigated and compared aiming to significantly reduce the network size so that it can run efficiently on resource-constrained edge devices such as wireless smart sensors. Further, depthwise separable convolution (DSC) is invoked, the integration of which with advanced structural pruning methods exhibited superior compression capability. Last but not least, quantization-aware training (QAT) is adopted for faster processing and lower memory and power consumption. The proposed edge intelligence framework will eventually lead to reduced network overload and latency. This will enable intelligent self-adaptation strategies to be employed to timely deal with a faulty sensor, minimizing the wasteful use of power, memory, and other resources in wireless smart sensors, increasing efficiency, and reducing maintenance costs for modern smart SHM systems. This study presents a theoretical foundation for the proposed framework, the validation of which through actual field trials is a scope for future work.

Convolutional Neural Network Based on Accelerator-Aware Pruning for Object Detection in Single-Shot Multibox Detector (싱글숏 멀티박스 검출기에서 객체 검출을 위한 가속 회로 인지형 가지치기 기반 합성곱 신경망 기법)

  • Kang, Hyeong-Ju
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.24 no.1
    • /
    • pp.141-144
    • /
    • 2020
  • Convolutional neural networks (CNNs) show high performance in computer vision tasks including object detection, but a lot of weight storage and computation is required. In this paper, a pruning scheme is applied to CNNs for object detection, which can remove much amount of weights with a negligible performance degradation. Contrary to the previous ones, the pruning scheme applied in this paper considers the base accelerator architecture. With the consideration, the pruned CNNs can be efficiently performed on an ASIC or FPGA accelerator. Even with the constrained pruning, the resulting CNN shows a negligible degradation of detection performance, less-than-1% point degradation of mAP on VOD0712 test set. With the proposed scheme, CNNs can be applied to objection dtection efficiently.

An Learning Algorithm to find the Optimized Network Structure in an Incremental Model (점증적 모델에서 최적의 네트워크 구조를 구하기 위한 학습 알고리즘)

  • Lee Jong-Chan;Cho Sang-Yeop
    • Journal of Internet Computing and Services
    • /
    • v.4 no.5
    • /
    • pp.69-76
    • /
    • 2003
  • In this paper we show a new learning algorithm for pattern classification. This algorithm considered a scheme to find a solution to a problem of incremental learning algorithm when the structure becomes too complex by noise patterns included in learning data set. Our approach for this problem uses a pruning method which terminates the learning process with a predefined criterion. In this process, an iterative model with 3 layer feedforward structure is derived from the incremental model by an appropriate manipulations. Notice that this network structure is not full-connected between upper and lower layers. To verify the effectiveness of pruning method, this network is retrained by EBP. From this results, we can find out that the proposed algorithm is effective, as an aspect of a system performence and the node number included in network structure.

  • PDF

Genetic Algorithm for Node P겨ning of Neural Networks (신경망의 노드 가지치기를 위한 유전 알고리즘)

  • Heo, Gi-Su;Oh, Il-Seok
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.46 no.2
    • /
    • pp.65-74
    • /
    • 2009
  • In optimizing the neural network structure, there are two methods of the pruning scheme and the constructive scheme. In this paper we use the pruning scheme to optimize neural network structure, and the genetic algorithm to find out its optimum node pruning. In the conventional researches, the input and hidden layers were optimized separately. On the contrary we attempted to optimize the two layers simultaneously by encoding two layers in a chromosome. The offspring networks inherit the weights from the parent. For teaming, we used the existing error back-propagation algorithm. In our experiment with various databases from UCI Machine Learning Repository, we could get the optimal performance when the network size was reduced by about $8{\sim}25%$. As a result of t-test the proposed method was shown better performance, compared with other pruning and construction methods through the cross-validation.

A Pruning Algorithm of Neural Networks Using Impact Factors (임팩트 팩터를 이용한 신경 회로망의 연결 소거 알고리즘)

  • 이하준;정승범;박철훈
    • Journal of the Institute of Electronics Engineers of Korea CI
    • /
    • v.41 no.2
    • /
    • pp.77-86
    • /
    • 2004
  • In general, small-sized neural networks, even though they show good generalization performance, tend to fail to team the training data within a given error bound, whereas large-sized ones learn the training data easily but yield poor generalization. Therefore, a way of achieving good generalization is to find the smallest network that can learn the data, called the optimal-sized neural network. This paper proposes a new scheme for network pruning with ‘impact factor’ which is defined as a multiplication of the variance of a neuron output and the square of its outgoing weight. Simulation results of function approximation problems show that the proposed method is effective in regression.

Autonomous Broadcast Pruning Scheme using Coverage Estimation in Wireless Ad Hoc Network (무선 Ad Hoc 망에서 영역 추정을 통한 ABP 브로드캐스트 기법)

  • Bae Ki chan;Kim Nam gi;Yoon Hyun soo
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.30 no.4B
    • /
    • pp.170-177
    • /
    • 2005
  • Due to the redundant rebroadcast packets, network-wide broadcasting is a costly operation in wireless mobile ad hoc networks. To reduce this redundancy, most of previous approaches implicitly or explicitly require periodic refreshing of neighborhood information which continuously imposes additional broadcast overheads. In this paper, we propose a practical broadcast pruning scheme based on the local prediction of a remained coverage area. As the proposed scheme uses only information available in the on-going broadcast process, it can minimize the overheads prevalent in previous approaches.

Restructuring a Feed-forward Neural Network Using Hidden Knowledge Analysis (학습된 지식의 분석을 통한 신경망 재구성 방법)

  • Kim, Hyeon-Cheol
    • Journal of KIISE:Software and Applications
    • /
    • v.29 no.5
    • /
    • pp.289-294
    • /
    • 2002
  • It is known that restructuring feed-forward neural network affects generalization capability and efficiency of the network. In this paper, we introduce a new approach to restructure a neural network using abstraction of the hidden knowledge that the network has teamed. This method involves extracting local rules from non-input nodes and aggregation of the rules into global rule base. The extracted local rules are used for pruning unnecessary connections of local nodes and the aggregation eliminates any possible redundancies arid inconsistencies among local rule-based structures. Final network is generated by the global rule-based structure. Complexity of the final network is much reduced, compared to a fully-connected neural network and generalization capability is improved. Empirical results are also shown.

Performance Analysis of Optimal Neural Network structural BPN based on character value of Hidden node (은닉노드의 특징 값을 기반으로 한 최적신경망 구조의 BPN성능분석)

  • 강경아;이기준;정채영
    • Journal of the Korea Society of Computer and Information
    • /
    • v.5 no.2
    • /
    • pp.30-36
    • /
    • 2000
  • The hidden node plays a role of the functional units that classifies the features of input pattern in the given question. Therefore, a neural network that consists of the number of a suitable optimum hidden node has be on the rise as a factor that has an important effect upon a result. However there is a problem that decides the number of hidden nodes based on back-propagation learning algorithm. If the number of hidden nodes is designated very small perfect learning is not done because the input pattern given cannot be classified enough. On the other hand, if designated a lot, overfitting occurs due to the unnecessary execution of operation and extravagance of memory point. So, the recognition rate is been law and the generality is fallen. Therefore, this paper suggests a method that decides the number of neural network node with feature information consisted of the parameter of learning algorithm. It excludes a node in the Pruning target, that has a maximum value among the feature value obtained and compares the average of the rest of hidden node feature value with the feature value of each hidden node, and then would like to improve the learning speed of neural network deciding the optimum structure of the multi-layer neural network as pruning the hidden node that has the feature value smaller than the average.

  • PDF

Adaptive Structure of Modular Wavelet Neural Network (모듈환된 웨이블렛 신경망의 적응 구조 설계)

  • 서재용;김성주;조현찬;전홍태
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.11 no.9
    • /
    • pp.782-787
    • /
    • 2001
  • In this paper, we propose an growing and pruning algorithm to design the adaptive structure of modular wavelet neural network(MWNN) with F-projection and geometric growing criterion. Geometric growing criterion consists of estimated error criterion considering local error and angel criterion which attempts to assign wavelet function that is nearly orthogonal to all other existing wavelet functions. There criteria provide a methodology that a network designer can constructs wavelet neural network according to one s intention. The proposed growing algorithm grows the module and the size of modules. Also, the pruning algorithm eliminates unnecessary node of module or module from constructed MWNN to overcome the problem due to localized characteristics of wavelet neural network which is used to modules of MWNN. We apply the proposed constructing algorithm of the adaptive structure of MWNN to approximation problems of 1-D function and 2-D function, and evaluate the effectiveness of the proposed algorithm.

  • PDF