DOI QR코드

DOI QR Code

Design of Efficient Big Data Collection Method based on Mass IoT devices

방대한 IoT 장치 기반 환경에서 효율적인 빅데이터 수집 기법 설계

  • Choi, Jongseok (Spartan Software Education Institute, Soongsil University) ;
  • Shin, Yongtae (School of Computing, Soongsil University)
  • Received : 2021.08.13
  • Accepted : 2021.08.20
  • Published : 2021.08.30

Abstract

Due to the development of IT technology, hardware technologies applied to IoT equipment have recently been developed, so smart systems using low-cost, high-performance RF and computing devices are being developed. However, in the infrastructure environment where a large amount of IoT devices are installed, big data collection causes a load on the collection server due to a bottleneck between the transmitted data. As a result, data transmitted to the data collection server causes packet loss and reduced data throughput. Therefore, there is a need for an efficient big data collection technique in an infrastructure environment where a large amount of IoT devices are installed. Therefore, in this paper, we propose an efficient big data collection technique in an infrastructure environment where a vast amount of IoT devices are installed. As a result of the performance evaluation, the packet loss and data throughput of the proposed technique are completed without loss of the transmitted file. In the future, the system needs to be implemented based on this design.

IT기술의 발달로 인해 최근 IoT 장비에 적용되는 하드웨어 기술이 저비용, 고성능 RF 및 연산장치를 사용한 스마트 시스템들로 변화되고 있다. 그러나 방대한 양의 IoT 장비들이 설치된 인프라 환경에서 빅데이터 수집은 전송되는 데이터간 병목현상으로 인해 수집 서버의 부하가 발생한다. 이로인해 데이터수집 서버로 전송되는 데이터는 패킷 손실 및 데이터 처리율 감소 현상이 발생한다. 따라서 방대한 양의 IoT 장비들이 설치된 인프라 환경에서 효율적인 빅데이터 수집 기법이 필요하다. 이에 본 논문에서는 방대한 양의 IoT 장비들이 설치된 인프라 환경에서 효율적인 빅데이터 수집 기법을 제안한다. 성능평가 결과, 제안하는 기법의 패킷 손실 및 데이터 처리율은 전송되는 파일의 손실없이 전송이 완료된다. 향후 본 설계를 기반으로 시스템이 구현이 필요하다.

Keywords

References

  1. K. I. Kim, J. S. Kim, "Big Data Processing and Performance Improvement for Ship Trajectory using MapReduce Technique", Journal of The Korea Society of Computer and Information, Vol. 24 No. 10, pp. 65-70, Oct, 2019
  2. K. Shvachko, H. Kuang, S. Radia, R. Chansler, "The Hadoop Distributed File System. In Mass Storage Systems and Technologies (MSST)," 2010 IEEE 26th symposium on IEEE, Vol.1, No.1, pp.1-10, 2010.
  3. B. H. Lee, D. M. Yang, "A Security Log Analysis System using Logstash based on Apache Elasticsearch", Journal of the Korea Institute of Information and Communication Engineering, Vol. 22, No. 2, pp.382-389, Feb, 2018 https://doi.org/10.6109/jkiice.2018.22.2.382
  4. G. W. Jin, "A Study on the Data Collection Methods based Hadoop Distributed Environment", Journal of the Korea Convergence Society, Vol. 7, No. 5, pp.1-6, Oct, 2016 https://doi.org/10.15207/JKCS.2016.7.5.001
  5. K. S. Noh, S. T. Park. K. H. Park, "Convergence Study on Big Data Competency Reference Model", Journal of Digital Convergence, Vol. 13, No. 3, pp.55-63, 2015 https://doi.org/10.14400/JDC.2015.13.3.55
  6. Y. H. Lee, J. H. Suh, "Big Data Platform for Utilizing and Analyzing Real-Time Sensing Information in Industrial Sites", The Korean Society for Creative Information Culture, Vol. 6, No. 1, pp.15-21, Apr, 2020
  7. V. Q. Nguyen, H. N. Nguyen, K. B. Kim, "Design of a Platform for Collecting and Analyzing Agricultural Big Data", Journal of Digital Contents Society, Vol. 18, No. 1, pp. 149-158, Feb. 2017 https://doi.org/10.9728/dcs.2017.18.1.149
  8. J. M. Moon, K. S. Shin, "Measurement of Latency and Uplink Throughput According to Number of NB-IoT Devices", The Journal of Korean Institute of Communications and Information Sciences ,Vol.44 No.06, pp.1188-1192