• Title/Summary/Keyword: IoT data compression

Search Result 16, Processing Time 0.024 seconds

In-memory Compression Scheme Based on Incremental Frequent Patterns for Graph Streams (그래프 스트림 처리를 위한 점진적 빈발 패턴 기반 인-메모리 압축 기법)

  • Lee, Hyeon-Byeong;Shin, Bo-Kyoung;Bok, Kyoung-Soo;Yoo, Jae-Soo
    • The Journal of the Korea Contents Association
    • /
    • v.22 no.1
    • /
    • pp.35-46
    • /
    • 2022
  • Recently, with the development of network technologies, as IoT and social network service applications have been actively used, a lot of graph stream data is being generated. In this paper, we propose a graph compression scheme that considers the stream graph environment by applying graph mining to the existing compression technique, which has been focused on compression rate and runtime. In this paper, we proposed Incremental frequent pattern based compression technique for graph streams. Since the proposed scheme keeps only the latest reference patterns, it increases the storage utilization and improves the query processing time. In order to show the superiority of the proposed scheme, various performance evaluations are performed in terms of compression rate and processing time compared to the existing method. The proposed scheme is faster than existing similar scheme when the number of duplicated data is large.

A Simulation Framework for Wireless Compressed Data Broadcast

  • Seokjin Im
    • International Journal of Advanced Culture Technology
    • /
    • v.11 no.2
    • /
    • pp.315-322
    • /
    • 2023
  • Intelligent IoT environments that accommodate a very large number of clients require technologies that provide secure information service regardless of the number of clients. Wireless data broadcast is an information service technique that ensures scalability to deliver data to all clients simultaneously regardless of the number of clients. In wireless data broadcasting, clients access the wireless channel linearly to explore the data, so the access time of clients is greatly affected by the broadcast cycle. Data compression-based data broadcasting can reduce the broadcast cycle and thus reduce client access time. Therefore, a simulation framework that can evaluate the performance of data broadcasting by applying different data compression algorithms is essential and important. In this paper, we propose a simulation framework to evaluate the performance of data broadcasting that can adopt data compression. We design the framework that enables to apply different data compression algorithms according to the data characteristics. In addition to evaluating the performance according to the data, the proposed framework can also evaluate the performance according to the data scheduling technique and the kind of queries the client wants to process. We implement the proposed framework and evaluate the performance of data broadcasting using the framework applying data compression algorithms to demonstrate the performances of data compression broadcasting.

Data Compression Method for Reducing Sensor Data Loss and Error in Wireless Sensor Networks (무선센서네트워크에서 센서 데이터 손실과 오류 감소를 위한 데이터 압축 방법)

  • Shin, DongHyun;Kim, Changhwa
    • Journal of Korea Multimedia Society
    • /
    • v.19 no.2
    • /
    • pp.360-374
    • /
    • 2016
  • Since WSNs (Wireless Sensor Networks) applied to their application areas such as smart home, smart factory, environment monitoring, etc., depend on sensor data, the sensor data is the most important among WSN components. The resources of each node consisting of WSN are extremely limited in energy, hardware and so on. Due to these limitation, communication failure probabilities become much higher and the communication failure causes data loss to occur. For this reason, this paper proposes 2MC (Maximum/Minimum Compression) that is a method to compress sensor data by selecting circular queue-based maximum/minimum sensor data values. Our proposed method reduces sensor data losses and value errors when they are recovered. Experimental results of 2MC method show the maximum/minimum 35% reduction efficiency in average sensor data accumulation error rate after the 3 times compression, comparing with CQP (Circular Queue Compression based on Period) after the compressed data recovering.

A hybrid deep neural network compression approach enabling edge intelligence for data anomaly detection in smart structural health monitoring systems

  • Tarutal Ghosh Mondal;Jau-Yu Chou;Yuguang Fu;Jianxiao Mao
    • Smart Structures and Systems
    • /
    • v.32 no.3
    • /
    • pp.179-193
    • /
    • 2023
  • This study explores an alternative to the existing centralized process for data anomaly detection in modern Internet of Things (IoT)-based structural health monitoring (SHM) systems. An edge intelligence framework is proposed for the early detection and classification of various data anomalies facilitating quality enhancement of acquired data before transmitting to a central system. State-of-the-art deep neural network pruning techniques are investigated and compared aiming to significantly reduce the network size so that it can run efficiently on resource-constrained edge devices such as wireless smart sensors. Further, depthwise separable convolution (DSC) is invoked, the integration of which with advanced structural pruning methods exhibited superior compression capability. Last but not least, quantization-aware training (QAT) is adopted for faster processing and lower memory and power consumption. The proposed edge intelligence framework will eventually lead to reduced network overload and latency. This will enable intelligent self-adaptation strategies to be employed to timely deal with a faulty sensor, minimizing the wasteful use of power, memory, and other resources in wireless smart sensors, increasing efficiency, and reducing maintenance costs for modern smart SHM systems. This study presents a theoretical foundation for the proposed framework, the validation of which through actual field trials is a scope for future work.

A Consistent Quality Bit Rate Control for the Line-Based Compression

  • Ham, Jung-Sik;Kim, Ho-Young;Lee, Seong-Won
    • IEIE Transactions on Smart Processing and Computing
    • /
    • v.5 no.5
    • /
    • pp.310-318
    • /
    • 2016
  • Emerging technologies such as the Internet of Things (IoT) and the Advanced Driver Assistant System (ADAS) often have image transmission functions with tough constraints, like low power and/or low delay, which require that they adopt line-based, low memory compression methods instead of existing frame-based image compression standards. Bit rate control in the conventional frame-based compression systems requires a lot of hardware resources when the scope of handled data falls at the frame level. On the other hand, attempts to reduce the heavy hardware resource requirement by focusing on line-level processing yield uneven image quality through the frame. In this paper, we propose a bit rate control that maintains consistency in image quality through the frame and improves the legibility of text regions. To find the line characteristics, the proposed bit rate control tests each line for ease of compression and the existence of text. Experiments on the proposed bit rate control show peak signal-to-noise ratios (PSNRs) similar to those of conventional bit rate controls, but with the use of significantly fewer hardware resources.

Analysis of Feature Map Compression Efficiency and Machine Task Performance According to Feature Frame Configuration Method (피처 프레임 구성 방안에 따른 피처 맵 압축 효율 및 머신 태스크 성능 분석)

  • Rhee, Seongbae;Lee, Minseok;Kim, Kyuheon
    • Journal of Broadcast Engineering
    • /
    • v.27 no.3
    • /
    • pp.318-331
    • /
    • 2022
  • With the recent development of hardware computing devices and software based frameworks, machine tasks using deep learning networks are expected to be utilized in various industrial fields and personal IoT devices. However, in order to overcome the limitations of high cost device for utilizing the deep learning network and that the user may not receive the results requested when only the machine task results are transmitted from the server, Collaborative Intelligence (CI) proposed the transmission of feature maps as a solution. In this paper, an efficient compression method for feature maps with vast data sizes to support the CI paradigm was analyzed and presented through experiments. This method increases redundancy by applying feature map reordering to improve compression efficiency in traditional video codecs, and proposes a feature map method that improves compression efficiency and maintains the performance of machine tasks by simultaneously utilizing image compression format and video compression format. As a result of the experiment, the proposed method shows 14.29% gain in BD-rate of BPP and mAP compared to the feature compression anchor of MPEG-VCM.