IEMEK Journal of Embedded Systems and Applications (대한임베디드공학회논문지)

Institute of Embedded Engineering of Korea (대한임베디드공학회)
권호 표지
DOI QR코드

DOI QR Code

A Study of Big Time Series Data Compression based on CNN Algorithm

CNN 기반 대용량 시계열 데이터 압축 기법연구

  • Received : 2022.09.21
  • Accepted : 2023.01.02
  • Published : 2023.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we implement a lossless compression technique for time-series data generated by IoT (Internet of Things) devices to reduce the disk spaces. The proposed compression technique reduces the size of the encoded data by selectively applying CNN (Convolutional Neural Networks) or Delta encoding depending on the situation in the Forecasting algorithm that performs prediction on time series data. In addition, the proposed technique sequentially performs zigzag encoding, splitting, and bit packing to increase the compression ratio. We showed that the proposed compression method has a compression ratio of up to 1.60 for the original data.

Keywords

Acknowledgement

본 연구는 중소벤처기업부의 규제자유특구혁신사업육성 지원에 의한 연구임 [P0020333].

References

  1. R. Krishnamurthi, A. Kumar, D. Gopinathan, A. Nayyar, B. Qureshi, "An Overview of IoT Sensor Data Processing, Fusion, and Analysis Techniques." Sensors, Vol 20, No. 21, pp. 6076, 2020.
  2. A. L. Diedrichs, F. Bromberg, D. Dujovne, K. Brun-Laguna, T. Watteyne, "Prediction of Frost Events Using Machine Learning and IoT Sensing Devices." IEEE Internet of Things Journal, Vol. 5, No. 6, pp. 4589-4597, 2018. https://doi.org/10.1109/JIOT.2018.2867333
  3. R. Akhter, S. A. Sofi, "Precision Agriculture Using IoT Data Analytics and Machine Learning." Journal of King Saud University-Computer and Information Sciences, Vol. 34, No. 8, pp. 5602-5618, 2021. https://doi.org/10.1016/j.jksuci.2021.05.013
  4. S. Ayvaz, K. Alpay, "Predictive Maintenance System for Production Lines in Manufacturing: A Machine Learning Approach Using IoT Data in Real-time." Expert Systems with Applications, Vol. 173, pp. 114598, 2021.
  5. J. Azar, A. Makhoul, M. Barhamgi, R. Couturier, "An Energy Efficient IoT Data Compression Approach for Edge Machine Learning." Future Generation Computer Systems, Vol. 96, pp. 168-175, 2019. https://doi.org/10.1016/j.future.2019.02.005
  6. H. M. Al-Kadhim, H. S. Al-Raweshidy, "Energy Efficient Data Compression in Cloud Based IoT." IEEE Sensors Journal, Vol. 21, No. 10, pp. 12212-12219, 2021. https://doi.org/10.1109/JSEN.2021.3064611
  7. G. Chiarot, C. Silvestri, "Time Series Compression Survey." ACM Computing Surveys (CSUR), Vol. 55, No. 10, pp. 1-32, 2021. https://doi.org/10.1145/3560814
  8. J. Azar, A. Makhoul, R. Couturier, J. Demerjian, "Robust IoT Time Series Classification with Data Compression and Deep Learning." Neurocomputing, Vol. 398, pp. 222-234, 2020. https://doi.org/10.1016/j.neucom.2020.02.097
  9. J. Azar, G. B. Tayeh, A. Makhoul, R. Couturier, "Efficient Lossy Compression for IoT Using SZ and Reconstruction with 1D U-Net." Mobile Networks and Applications, Vol. 27, pp. 984-996, 2022. https://doi.org/10.1007/s11036-022-01918-6
  10. T. Pelkonen, S. Franklin, J. Teller, P. Cavallaro, Q. Huang, J. Meza, K. Veeraraghavan, "Gorilla: A Fast, Scalable, in-memory Time Series Database." Proceedings of the VLDB Endowment, Vol. 8, No. 12, pp. 1816-1827, 2015. https://doi.org/10.14778/2824032.2824078
  11. D. Blalock, S. Madden, J. Guttag, "Sprintz: Time Series Compression for the Internet of Things." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Vol. 2, No. 3, pp 1-23, 2018. https://doi.org/10.1145/3264903
  12. J. Burgues, J. M.l Jimenez-Soto, S. Marco, "Estimation of the Limit of Detection in Semiconductor Gas Sensors Through Linearized Calibration Models." Analytica Chimica Acta, Vol. 1013, pp. 13-25, 2018. https://doi.org/10.1016/j.aca.2018.01.062
  13. S. Zhang, B. Guo, A. Dong, J. He, Z. Xu, S. X. Chen, "Cautionary Tales on Air-quality Improvement in Beijing." Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 473:20170457, 2017.