• Title/Summary/Keyword: Big data collection

Search Result 340, Processing Time 0.026 seconds

Implementation of a data collection system for big data analysis and learning based on infant body temperature data (영유아 체온 데이터 기반 빅데이터 분석 및 학습을 위한 데이터 수집 시스템 구현)

  • Lee, Hyoun-Sup;Heo, Gyeongyong
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2021.05a
    • /
    • pp.577-578
    • /
    • 2021
  • Recently, artificial intelligence systems are being used in various fields. The accuracy of the decision algorithm of artificial intelligence is greatly affected by the amount of learning and the accuracy of the learning data. In the case of the amount of learning, a large amount of data is required because it has a decisive effect on the performance of AI. In this paper, we propose a data collection system for constructing a system that analyzes future conditions and changes in infants' conditions based on the body temperature data of infants and toddlers. The proposed system is a system that collects and transmits data, and it is believed that it can minimize the resource consumption of the server system in existing big data analysis and training data construction.

  • PDF

Self-organization Scheme of WSNs with Mobile Sensors and Mobile Multiple Sinks for Big Data Computing

  • Shin, Ahreum;Ryoo, Intae;Kim, Seokhoon
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.14 no.3
    • /
    • pp.943-961
    • /
    • 2020
  • With the advent of IoT technology and Big Data computing, the importance of WSNs (Wireless Sensor Networks) has been on the rise. For energy-efficient and collection-efficient delivery of any sensed data, lots of novel wireless medium access control (MAC) protocols have been proposed and these MAC schemes are the basis of many IoT systems that leads the upcoming fourth industrial revolution. WSNs play a very important role in collecting Big Data from various IoT sensors. Also, due to the limited amount of battery driving the sensors, energy-saving MAC technologies have been recently studied. In addition, as new IoT technologies for Big Data computing emerge to meet different needs, both sensors and sinks need to be mobile. To guarantee stability of WSNs with dynamic topologies as well as frequent physical changes, the existing MAC schemes must be tuned for better adapting to the new WSN environment which includes energy-efficiency and collection-efficiency of sensors, coverage of WSNs and data collecting methods of sinks. To address these issues, in this paper, a self-organization scheme for mobile sensor networks with mobile multiple sinks has been proposed and verified to adapt both mobile sensors and multiple sinks to 3-dimensional group management MAC protocol. Performance evaluations show that the proposed scheme outperforms the previous schemes in terms of the various usage cases. Therefore, the proposed self-organization scheme might be adaptable for various computing and networking environments with big data.

Development of Big-data Management Platform Considering Docker Based Real Time Data Connecting and Processing Environments (도커 기반의 실시간 데이터 연계 및 처리 환경을 고려한 빅데이터 관리 플랫폼 개발)

  • Kim, Dong Gil;Park, Yong-Soon;Chung, Tae-Yun
    • IEMEK Journal of Embedded Systems and Applications
    • /
    • v.16 no.4
    • /
    • pp.153-161
    • /
    • 2021
  • Real-time access is required to handle continuous and unstructured data and should be flexible in management under dynamic state. Platform can be built to allow data collection, storage, and processing from local-server or multi-server. Although the former centralize method is easy to control, it creates an overload problem because it proceeds all the processing in one unit, and the latter distributed method performs parallel processing, so it is fast to respond and can easily scale system capacity, but the design is complex. This paper provides data collection and processing on one platform to derive significant insights from various data held by an enterprise or agency in the latter manner, which is intuitively available on dashboards and utilizes Spark to improve distributed processing performance. All service utilize dockers to distribute and management. The data used in this study was 100% collected from Kafka, showing that when the file size is 4.4 gigabytes, the data processing speed in spark cluster mode is 2 minute 15 seconds, about 3 minutes 19 seconds faster than the local mode.

A Study on Big Data Analysis of Public Library in Busan: Based on the Library Collection/Circulation Data (부산지역 공공도서관의 빅데이터 분석 연구 - 도서관 정보나루 장서/대출데이터를 중심으로 -)

  • Lee, Soon-Young;Lee, Soo-Sang
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.55 no.4
    • /
    • pp.89-114
    • /
    • 2021
  • This study analyzed the previous studies and utilization cases on library big data, and based on this, analyzed the collection/circulation data of the library big data platform and tried to derive meaningful analysis results. And five analysis indicators were selected: the increase rate of collections by annual, the composition of collections by subject, the composition of unborrowed collections by subject, the rate of borrowed collections, and use factor by subject. The analysis data is 6,722,603 cases of collection/circulation data from 33 public libraries in Busan. The main analysis results are as follows. First, it was found that the gap in the number of circulation was larger than the number of collection in the 33 public libraries. Second, the annual increase rate of collections also showed a clear decline. Third, each library showed a similar pattern in the composition of both the collections and the unborrowed collections by subject. Fourth, it was found that users' circulation were very different by subject and library. Fifth, in most libraries, the rate of circulation of collections and use factor in the natural science field were the highest.

Neo-Chinese Style Furniture Design Based on Semantic Analysis and Connection

  • Ye, Jialei;Zhang, Jiahao;Gao, Liqian;Zhou, Yang;Liu, Ziyang;Han, Jianguo
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.16 no.8
    • /
    • pp.2704-2719
    • /
    • 2022
  • Lately, neo-Chinese style furniture has been frequently noticed by product design professionals for the big part it played in promoting traditional Chinese culture. This article is an attempt to use big data semantic analysis method to provide effective design research method for neo-Chinese furniture design. By using big data mining program TEXTOM for big data collection and analysis, the data obtained from typical websites in a set time period will be sorted and analyzed. On the basis of "neo-Chinese furniture" samples, key data will be compared, classification analysis of overall data, and horizontal analysis of typical data will be performed by the methods of word frequency analysis, connection centrality analysis, and TF-IDF analysis. And we tried to summarize according to the related views and theories of the design. The research results show that the results of data analysis are close to the relevant definitions of design. The core high-frequency vocabulary obtained under data analysis, such as popular, furniture, modern, etc., can provide a reasonable and effective focus of attention for the designs. The result obtained through the systematic sorting and summary of the data can be a reliable guidance in the direction of our design. This research attempted to introduce related big data mining semantic analysis methods into the product design industry, to supply scientific and objective data and channels for studies on design, and to provide a case on the practical application of big data analysis in the industry.

Designing Bigdata Platform for Multi-Source Maritime Information

  • Junsang Kim
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.1
    • /
    • pp.111-119
    • /
    • 2024
  • In this paper, we propose a big data platform that can collect information from various sources collected at ocean. Currently operating ocean-related big data platforms are focused on storing and sharing created data, and each data provider is responsible for data collection and preprocessing. There are high costs and inefficiencies in collecting and integrating data in a marine environment using communication networks that are poor compared to those on land, making it difficult to implement related infrastructure. In particular, in fields that require real-time data collection and analysis, such as weather information, radar and sensor data, a number of issues must be considered compared to land-based systems, such as data security, characteristics of organizations and ships, and data collection costs, in addition to communication network issues. First, this paper defines these problems and presents solutions. In order to design a big data platform that reflects this, we first propose a data source, hierarchical MEC, and data flow structure, and then present an overall platform structure that integrates them all.

For Improving Security Log Big Data Analysis Efficiency, A Firewall Log Data Standard Format Proposed (보안로그 빅데이터 분석 효율성 향상을 위한 방화벽 로그 데이터 표준 포맷 제안)

  • Bae, Chun-sock;Goh, Sung-cheol
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.30 no.1
    • /
    • pp.157-167
    • /
    • 2020
  • The big data and artificial intelligence technology, which has provided the foundation for the recent 4th industrial revolution, has become a major driving force in business innovation across industries. In the field of information security, we are trying to develop and improve an intelligent security system by applying these techniques to large-scale log data, which has been difficult to find effective utilization methods before. The quality of security log big data, which is the basis of information security AI learning, is an important input factor that determines the performance of intelligent security system. However, the difference and complexity of log data by various product has a problem that requires excessive time and effort in preprocessing big data with poor data quality. In this study, we research and analyze the cases related to log data collection of various firewall. By proposing firewall log data collection format standard, we hope to contribute to the development of intelligent security systems based on security log big data.

A Business Application of the Business Intelligence and the Big Data Analytics (비즈니스 인텔리전스와 빅데이터 분석의 비즈니스 응용)

  • Lee, Ki-Kwang;Kim, Tae-Hwan
    • Journal of Korean Society of Industrial and Systems Engineering
    • /
    • v.42 no.4
    • /
    • pp.84-90
    • /
    • 2019
  • Lately, there have been tremendous shifts in the business technology landscape. Advances in cloud technology and mobile applications have enabled businesses and IT users to interact in entirely new ways. One of the most rapidly growing technologies in this sphere is business intelligence, and associated concepts such as big data and data mining. BI is the collection of systems and products that have been implemented in various business practices, but not the information derived from the systems and products. On the other hand, big data has come to mean various things to different people. When comparing big data vs business intelligence, some people use the term big data when referring to the size of data, while others use the term in reference to specific approaches to analytics. As the volume of data grows, businesses will also ask more questions to better understand the data analytics process. As a result, the analysis team will have to keep up with the rising demands on the infrastructure that supports analytics applications brought by these additional requirements. It's also a good way to ascertain if we have built a valuable analysis system. Thus, Business Intelligence and Big Data technology can be adapted to the business' changing requirements, if they prove to be highly valuable to business environment.

A Study on The Real-Time Data Collection/Analysis/Processing Intelligent IoT (실시간 데이터 수집/분석/처리를 위한 지능형 IoT)

  • Kim, Hee-Chul
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.14 no.2
    • /
    • pp.317-322
    • /
    • 2019
  • This study is based on big big data base for real-time collection/analysis/processing of data, creative analysis of data assets, and intelligent processing system based on IoT, which can measure distribution phase in real time. The mobile terminal uses the SDK of the provided device to measure the data information on the consumption of specific seafood production and distribution. We use the oneM2M protocol to store various kinds of information needed for seafood production, and implement a DB Server and a system that allows the administrator to manage the system using the UI.

A Study on the Promotion of Yakseon Food Using Big Data

  • LEE, JINHO;KIM, AE SOOK;Hwang, Chi-Gon;Ryu, Gi Hwan
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.14 no.4
    • /
    • pp.41-46
    • /
    • 2022
  • The purpose of this study is to confirm and analyze the impact on consumers through big data keyword analysis on weak food. For data collection, web documents, blogs, news, cafes, intellectuals, academic information, and Google Web, news, and Facebook provided by Naver and Daum were used as analysis targets. The data analysis period was set from January 2018 to December 2021. For data collection and analysis, the frequency and matrix of keywords were extracted through Textom, a social matrix site, and the relationship and connection centrality between keywords were analyzed and visualized using the Netdraw function among UCINET6 programs. In addition, CONCOR analysis was conducted to derive clusters for similar keywords. As a result of analyzing yakseon food with keywords, a total of 35,985 cases of collected data were derived. Through this, it was confirmed that medicinal food affects consumers. Furthermore, if a business model is created and developed through yakseon food, it will be possible to lead the popularization of yakseon food.