• Title/Summary/Keyword: network pruning

Search Result 82, Processing Time 0.027 seconds

Elimination of Redundant Input Information and Parameters during Neural Network Training (신경망 학습 과정중 불필요한 입력 정보 및 파라미터들의 제거)

  • Won, Yong-Gwan;Park, Gwang-Gyu
    • The Transactions of the Korea Information Processing Society
    • /
    • v.3 no.3
    • /
    • pp.439-448
    • /
    • 1996
  • Extraction and selection of the informative features play a central role in pattern recognition. This paper describes a modified back-propagation algorithm that performs selection of the informative features and trains a neural network simultaneously. The algorithm is mainly composed of three repetitive steps : training, connection pruning, and input unit elimination. Afer initial training, the connections that have small magnitude are first pruned. Any unit that has a small number of connections to the hidden units is deleted,which is equivalent to excluding the feature corresponding to that unit.If the error increases,the network is retraned,again followed by connection pruning and input unit elimination.As a result,the algorithm selects the most im-portant features in the measurement space without a transformation to another space.Also,the selected features are the most-informative ones for the classification,because feature selection is tightly coupled with the classifi-cation performance.This algorithm helps avoid measurement of redundant or less informative features,which may be expensive.Furthermore,the final network does not include redundant parameters,i.e.,weights and biases,that may cause degradation of classification performance.In applications,the algorithm preserves the most informative features and significantly reduces the dimension of the feature vectors whiout performance degradation.

  • PDF

A Study on Compression of Connections in Deep Artificial Neural Networks (인공신경망의 연결압축에 대한 연구)

  • Ahn, Heejune
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.22 no.5
    • /
    • pp.17-24
    • /
    • 2017
  • Recently Deep-learning, Technologies using Large or Deep Artificial Neural Networks, have Shown Remarkable Performance, and the Increasing Size of the Network Contributes to its Performance Improvement. However, the Increase in the Size of the Neural Network Leads to an Increase in the Calculation Amount, which Causes Problems Such as Circuit Complexity, Price, Heat Generation, and Real-time Restriction. In This Paper, We Propose and Test a Method to Reduce the Number of Network Connections by Effectively Pruning the Redundancy in the Connection and Showing the Difference between the Performance and the Desired Range of the Original Neural Network. In Particular, we Proposed a Simple Method to Improve the Performance by Re-learning and to Guarantee the Desired Performance by Allocating the Error Rate per Layer in Order to Consider the Difference of each Layer. Experiments have been Performed on a Typical Neural Network Structure such as FCN (full connection network) and CNN (convolution neural network) Structure and Confirmed that the Performance Similar to that of the Original Neural Network can be Obtained by Only about 1/10 Connection.

Complete and Incomplete Observability Analysis by Optimal PMU Placement Techniques of a Network

  • Krishna, K. Bala;Rosalina, K. Mercy;Ramaraj, N.
    • Journal of Electrical Engineering and Technology
    • /
    • v.13 no.5
    • /
    • pp.1814-1820
    • /
    • 2018
  • State estimation of power systems has become vital in recent days of power operation and control. SCADA and EMS are intended for the state estimation and to communicate and monitor the systems which are operated at specified time. Although various methods are used we can achieve the better results by using PMU technique. On placing the PMU, operating time is reduced and making the performance reliable. In this paper, PMU placement is done in two ways. Those are 'optimal technique with pruning operation' and 'depth of unobservability' considering incomplete and complete observability of a network. By Depth of Unobservability Number of PMUs are reduced to attain Observability of the network. Proposed methods are tested on IEEE 14, 30, 57, SR-system and Sub systems (1, 2) with bus size of 270 and 444 buses. Along with achieving complete observability analysis, single PMU loss condition is also achieved.

CASPER: Congestion Aware Selection of Path with Efficient Routing in Multimedia Networks

  • Obaidat, Mohammad S.;Dhurandher, Sanjay K.;Diwakar, Khushboo
    • Journal of Information Processing Systems
    • /
    • v.7 no.2
    • /
    • pp.241-260
    • /
    • 2011
  • In earlier days, most of the data carried on communication networks was textual data requiring limited bandwidth. With the rise of multimedia and network technologies, the bandwidth requirements of data have increased considerably. If a network link at any time is not able to meet the minimum bandwidth requirement of data, data transmission at that path becomes difficult, which leads to network congestion. This causes delay in data transmission and might also lead to packet drops in the network. The retransmission of these lost packets would aggravate the situation and jam the network. In this paper, we aim at providing a solution to the problem of network congestion in mobile ad hoc networks [1, 2] by designing a protocol that performs routing intelligently and minimizes the delay in data transmission. Our Objective is to move the traffic away from the shortest path obtained by a suitable shortest path calculation algorithm to a less congested path so as to minimize the number of packet drops during data transmission and to avoid unnecessary delay. For this we have proposed a protocol named as Congestion Aware Selection Of Path With Efficient Routing (CASPER). Here, a router runs the shortest path algorithm after pruning those links that violate a given set of constraints. The proposed protocol has been compared with two link state protocols namely, OSPF [3, 4] and OLSR [5, 6, 7, 8].The results achieved show that our protocol performs better in terms of network throughput and transmission delay in case of bulky data transmission.

Mobile Volume Rendering System for Client-Server Environment (클라이언트 서버 기반 모바일 볼륨 가시화 시스템)

  • Lee, Woongkyu;Kye, Heewon
    • Journal of the Korea Computer Graphics Society
    • /
    • v.21 no.3
    • /
    • pp.17-26
    • /
    • 2015
  • In this paper, we explain a volume rendering system for client-server environment. A single GPU-equipped PC works as a server which is based on the ideas that only a few concurrent users use a volume rendering system in a small hospital. As the clients, we used Android mobile devices such as smart phones. User events are transformed to rendering requests by the client application. When the server receives a rendering request, it renders the volume using the GPU. The rendered image is compressed to JPEG or PNG format so that we can save network bandwidth and reduce transfer time. In addition, we perform an event pruning method while a user is dragging the touch to enhance latency. The server compensates the pruning by interpolating the touch positions. As the result, real-time volume rendering is possible for 5 concurrent users on single GPU-equipped commodity hardware.

An Improved Signature Hashing-based Pattern Matching for High Performance IPS (고성능 침입방지 시스템을 위해 개선한 시그니처 해싱 기반 패턴 매칭 기법)

  • Lee, Young-Sil;Kim, Nack-Hyun;Lee, Hoon-Jae
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2010.10a
    • /
    • pp.434-437
    • /
    • 2010
  • NIPS(Network Intrusion Prevention System) is in line at the end of the external and internal networks which performed two kinds of action: Signature-based filtering and anomaly detection and prevention-based on self-learning. Among them, a signature-based filtering is well known to defend against attacks. By using signature-based filtering, intrusion prevention system passing a payload of packets is compared with attack patterns which are signature. If match, the packet is discard. However, when there is packet delay, it will increase the required pattern matching time as the number of signature is increasing whenever there is delay occur. Therefore, to ensure the performance of IPS, we needed more efficient pattern matching algorithm for high-performance ISP. To improve the performance of pattern matching the most important part is to reduce the number of comparisons signature rules and the packet whenever the packets arrive. In this paper, we propose an improve signature hashing-based pattern matching method. We use tuple pruning algorithm with Bloom filters, which effectively remove unnecessary tuples. Unlike other existing signature hashing-based IPS, our proposed method to improve the performance of IPS.

  • PDF

Implementation of FPGA-based Accelerator for GRU Inference with Structured Compression (구조적 압축을 통한 FPGA 기반 GRU 추론 가속기 설계)

  • Chae, Byeong-Cheol
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.26 no.6
    • /
    • pp.850-858
    • /
    • 2022
  • To deploy Gate Recurrent Units (GRU) on resource-constrained embedded devices, this paper presents a reconfigurable FPGA-based GRU accelerator that enables structured compression. Firstly, a dense GRU model is significantly reduced in size by hybrid quantization and structured top-k pruning. Secondly, the energy consumption on external memory access is greatly reduced by the proposed reuse computing pattern. Finally, the accelerator can handle a structured sparse model that benefits from the algorithm-hardware co-design workflows. Moreover, inference tasks can be flexibly performed using all functional dimensions, sequence length, and number of layers. Implemented on the Intel DE1-SoC FPGA, the proposed accelerator achieves 45.01 GOPs in a structured sparse GRU network without batching. Compared to the implementation of CPU and GPU, low-cost FPGA accelerator achieves 57 and 30x improvements in latency, 300 and 23.44x improvements in energy efficiency, respectively. Thus, the proposed accelerator is utilized as an early study of real-time embedded applications, demonstrating the potential for further development in the future.

A Lightweight Pedestrian Intrusion Detection and Warning Method for Intelligent Traffic Security

  • Yan, Xinyun;He, Zhengran;Huang, Youxiang;Xu, Xiaohu;Wang, Jie;Zhou, Xiaofeng;Wang, Chishe;Lu, Zhiyi
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.12
    • /
    • pp.3904-3922
    • /
    • 2022
  • As a research hotspot, pedestrian detection has a wide range of applications in the field of computer vision in recent years. However, current pedestrian detection methods have problems such as insufficient detection accuracy and large models that are not suitable for large-scale deployment. In view of these problems mentioned above, a lightweight pedestrian detection and early warning method using a new model called you only look once (Yolov5) is proposed in this paper, which utilizing advantages of Yolov5s model to achieve accurate and fast pedestrian recognition. In addition, this paper also optimizes the loss function of the batch normalization (BN) layer. After sparsification, pruning and fine-tuning, got a lot of optimization, the size of the model on the edge of the computing power is lower equipment can be deployed. Finally, from the experimental data presented in this paper, under the training of the road pedestrian dataset that we collected and processed independently, the Yolov5s model has certain advantages in terms of precision and other indicators compared with traditional single shot multiBox detector (SSD) model and fast region-convolutional neural network (Fast R-CNN) model. After pruning and lightweight, the size of training model is greatly reduced without a significant reduction in accuracy, and the final precision reaches 87%, while the model size is reduced to 7,723 KB.

GEP-based Framework for Immune-Inspired Intrusion Detection

  • Tang, Wan;Peng, Limei;Yang, Ximin;Xie, Xia;Cao, Yang
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.4 no.6
    • /
    • pp.1273-1293
    • /
    • 2010
  • Immune-inspired intrusion detection is a promising technology for network security, and well known for its diversity, adaptation, self-tolerance, etc. However, scalability and coverage are two major drawbacks of the immune-inspired intrusion detection systems (IIDSes). In this paper, we propose an IIDS framework, named GEP-IIDS, with improved basic system elements to address these two problems. First, an additional bio-inspired technique, gene expression programming (GEP), is introduced in detector (corresponding to detection rules) representation. In addition, inspired by the avidity model of immunology, new avidity/affinity functions taking the priority of attributes into account are given. Based on the above two improved elements, we also propose a novel immune algorithm that is capable of integrating two bio-inspired mechanisms (i.e., negative selection and positive selection) by using a balance factor. Finally, a pruning algorithm is given to reduce redundant detectors that consume footprint and detection time but do not contribute to improving performance. Our experimental results show the feasibility and effectiveness of our solution to handle the scalability and coverage problems of IIDS.

Efficient Query Retrieval from Social Data in Neo4j using LIndex

  • Mathew, Anita Brigit
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.12 no.5
    • /
    • pp.2211-2232
    • /
    • 2018
  • The unstructured and semi-structured big data in social network poses new challenges in query retrieval. This requirement needs to be met by introducing quality retrieval time measures like indexing. Due to the huge volume of data storage, there originate the need for efficient index algorithms to promote query processing. However, conventional algorithms fail to index the huge amount of frequently obtained information in real time and fall short of providing scalable indexing service. In this paper, a new LIndex algorithm, which is a heuristic on Lucene is built on Neo4jHA architecture that holds the social network Big data. LIndex is a flexible and simplified adaptive indexing scheme that ascendancy decomposed shortest paths around term neighbors as basic indexing unit. This newfangled index proves to be effectual in query space pruning of graph database Neo4j, scalable in index construction and deployment. A graph query is processed and optimized beyond the traditional Lucene in a time-based manner to a more efficient path method in LIndex. This advanced algorithm significantly reduces query fetch without compromising the quality of results in time. The experiments are conducted to confirm the efficiency of the proposed query retrieval in Neo4j graph NoSQL database.