• Title/Summary/Keyword: Big Data Environment

Search Result 959, Processing Time 0.036 seconds

Blockchain and AI-based big data processing techniques for sustainable agricultural environments (지속가능한 농업 환경을 위한 블록체인과 AI 기반 빅 데이터 처리 기법)

  • Yoon-Su Jeong
    • Advanced Industrial SCIence
    • /
    • v.3 no.2
    • /
    • pp.17-22
    • /
    • 2024
  • Recently, as the ICT field has been used in various environments, it has become possible to analyze pests by crops, use robots when harvesting crops, and predict by big data by utilizing ICT technologies in a sustainable agricultural environment. However, in a sustainable agricultural environment, efforts to solve resource depletion, agricultural population decline, poverty increase, and environmental destruction are constantly being demanded. This paper proposes an artificial intelligence-based big data processing analysis method to reduce the production cost and increase the efficiency of crops based on a sustainable agricultural environment. The proposed technique strengthens the security and reliability of data by processing big data of crops combined with AI, and enables better decision-making and business value extraction. It can lead to innovative changes in various industries and fields and promote the development of data-oriented business models. During the experiment, the proposed technique gave an accurate answer to only a small amount of data, and at a farm site where it is difficult to tag the correct answer one by one, the performance similar to that of learning with a large amount of correct answer data (with an error rate within 0.05) was found.

A Study on Prediction of Traffic Volume Using Road Management Big Data

  • Sung, Hongki;Chong, Kyusoo
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.33 no.6
    • /
    • pp.589-594
    • /
    • 2015
  • In reflection of road expansion and increasing use rates, interest has blossomed in predicting driving environment. In addition, a gigantic scale of big data is applied to almost every area around the world. Recently, technology development is being promoted in the area of road traffic particularly for traffic information service and analysis system in utilization of big data. This study examines actual cases of road management systems and road information analysis technologies, home and abroad. Based on the result, the limitations of existing technologies and road management systems are analyzed. In this study, a development direction and expected effort of the prediction of road information are presented. This study also examines regression analysis about relationship between guide name and traffic volume. According to the development of driving environment prediction platform, it will be possible to serve more reliable road information and also it will make safe and smart road infrastructures.

Analysis on Major Factors for Analysis & Application of Big Data in Electrical Commercial System (전자상거래 시스템에서 빅 데이터의 분석 및 결과 활용에 미치는 영향요소 분석)

  • Yang, Hoo-Youl;Na, Cheol-Hun
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2016.05a
    • /
    • pp.373-375
    • /
    • 2016
  • Analyze the Big Data become a hot issue because of Smart environment, the amount of data in the world has been exploding. Result of application makes a good use of Analysis and applicate of the big data, is play an important part in application area (finance, circulation, manufacturing, disaster etc.) This paper presents an influence element for data analysis and its practical use based in result of maturity in Business process of Big Data in Electrical Commercial system.

  • PDF

Implementing a Sustainable Decision-Making Environment - Cases for GIS, BIM, and Big Data Utilization -

  • Kim, Hwan-Yong
    • Journal of KIBIM
    • /
    • v.6 no.3
    • /
    • pp.24-33
    • /
    • 2016
  • Planning occurs from day-to-day, small-scale decisions to large-scale infrastructure investment decisions. For that reason, various attempts have been made to appropriately assist decision-making process and its optimization. Lately, initiation of a large amount of data, also known as big data has received great attention from diverse disciplines because of versatility and adoptability in its use and possibility to generate new information. Accordingly, implementation of big data and other information management systems, such as geographic information systems (GIS) and building information modeling (BIM) have received enough attention to establish each of its own profession and other associated activities. In this extent, this study illustrates a series of big data implementation cases that can provide a lesson to urban planning domain. In specific, case studies analyze how data was used to extract the most optimized solution and what aspects could be helpful in relation to planning decisions. Also, important notions about GIS and its application in various urban cases are examined.

Big Data Based Urban Transportation Analysis for Smart Cities - Machine Learning Based Traffic Prediction by Using Urban Environment Data - (도시 빅데이터를 활용한 스마트시티의 교통 예측 모델 - 환경 데이터와의 상관관계 기계 학습을 통한 예측 모델의 구축 및 검증 -)

  • Jang, Sun-Young;Shin, Dong-Youn
    • Journal of KIBIM
    • /
    • v.8 no.3
    • /
    • pp.12-19
    • /
    • 2018
  • The research aims to find implications of machine learning and urban big data as a way to construct the flexible transportation network system of smart city by responding the urban context changes. This research deals with a problem that existing a bus headway model is difficult to respond urban situations in real-time. Therefore, utilizing the urban big data and machine learning prototyping tool in weathers, traffics, and bus statues, this research presents a flexible headway model to predict bus delay and analyze the result. The prototyping model is composed by real-time data of buses. The data is gathered through public data portals and real time Application Program Interface (API) by the government. These data are fundamental resources to organize interval pattern models of bus operations as traffic environment factors (road speeds, station conditions, weathers, and bus information of operating in real-time). The prototyping model is implemented by the machine learning tool (RapidMiner Studio) and conducted several tests for bus delays prediction according to specific circumstances. As a result, possibilities of transportation system are discussed for promoting the urban efficiency and the citizens' convenience by responding to urban conditions.

Big data platform for health monitoring systems of multiple bridges

  • Wang, Manya;Ding, Youliang;Wan, Chunfeng;Zhao, Hanwei
    • Structural Monitoring and Maintenance
    • /
    • v.7 no.4
    • /
    • pp.345-365
    • /
    • 2020
  • At present, many machine leaning and data mining methods are used for analyzing and predicting structural response characteristics. However, the platform that combines big data analysis methods with online and offline analysis modules has not been used in actual projects. This work is dedicated to developing a multifunctional Hadoop-Spark big data platform for bridges to monitor and evaluate the serviceability based on structural health monitoring system. It realizes rapid processing, analysis and storage of collected health monitoring data. The platform contains offline computing and online analysis modules, using Hadoop-Spark environment. Hadoop provides the overall framework and storage subsystem for big data platform, while Spark is used for online computing. Finally, the big data Hadoop-Spark platform computational performance is verified through several actual analysis tasks. Experiments show the Hadoop-Spark big data platform has good fault tolerance, scalability and online analysis performance. It can meet the daily analysis requirements of 5s/time for one bridge and 40s/time for 100 bridges.

Design of GlusterFS Based Big Data Distributed Processing System in Smart Factory (스마트 팩토리 환경에서의 GlusterFS 기반 빅데이터 분산 처리 시스템 설계)

  • Lee, Hyeop-Geon;Kim, Young-Woon;Kim, Ki-Young;Choi, Jong-Seok
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.11 no.1
    • /
    • pp.70-75
    • /
    • 2018
  • Smart Factory is an intelligent factory that can enhance productivity, quality, customer satisfaction, etc. by applying information and communications technology to the entire production process including design & development, manufacture, and distribution & logistics. The precise amount of data generated in a smart factory varies depending on the factory's size and state of facilities. Regardless, it would be difficult to apply traditional production management systems to a smart factory environment, as it generates vast amounts of data. For this reason, the need for a distributed big-data processing system has risen, which can process a large amount of data. Therefore, this article has designed a Gluster File System (GlusterFS)-based distributed big-data processing system that can be used in a smart factory environment. Compared to existing distributed processing systems, the proposed distributed big-data processing system reduces the system load and the risk of data loss through the distribution and management of network traffic.

Design of Client-Server Model For Effective Processing and Utilization of Bigdata (빅데이터의 효과적인 처리 및 활용을 위한 클라이언트-서버 모델 설계)

  • Park, Dae Seo;Kim, Hwa Jong
    • Journal of Intelligence and Information Systems
    • /
    • v.22 no.4
    • /
    • pp.109-122
    • /
    • 2016
  • Recently, big data analysis has developed into a field of interest to individuals and non-experts as well as companies and professionals. Accordingly, it is utilized for marketing and social problem solving by analyzing the data currently opened or collected directly. In Korea, various companies and individuals are challenging big data analysis, but it is difficult from the initial stage of analysis due to limitation of big data disclosure and collection difficulties. Nowadays, the system improvement for big data activation and big data disclosure services are variously carried out in Korea and abroad, and services for opening public data such as domestic government 3.0 (data.go.kr) are mainly implemented. In addition to the efforts made by the government, services that share data held by corporations or individuals are running, but it is difficult to find useful data because of the lack of shared data. In addition, big data traffic problems can occur because it is necessary to download and examine the entire data in order to grasp the attributes and simple information about the shared data. Therefore, We need for a new system for big data processing and utilization. First, big data pre-analysis technology is needed as a way to solve big data sharing problem. Pre-analysis is a concept proposed in this paper in order to solve the problem of sharing big data, and it means to provide users with the results generated by pre-analyzing the data in advance. Through preliminary analysis, it is possible to improve the usability of big data by providing information that can grasp the properties and characteristics of big data when the data user searches for big data. In addition, by sharing the summary data or sample data generated through the pre-analysis, it is possible to solve the security problem that may occur when the original data is disclosed, thereby enabling the big data sharing between the data provider and the data user. Second, it is necessary to quickly generate appropriate preprocessing results according to the level of disclosure or network status of raw data and to provide the results to users through big data distribution processing using spark. Third, in order to solve the problem of big traffic, the system monitors the traffic of the network in real time. When preprocessing the data requested by the user, preprocessing to a size available in the current network and transmitting it to the user is required so that no big traffic occurs. In this paper, we present various data sizes according to the level of disclosure through pre - analysis. This method is expected to show a low traffic volume when compared with the conventional method of sharing only raw data in a large number of systems. In this paper, we describe how to solve problems that occur when big data is released and used, and to help facilitate sharing and analysis. The client-server model uses SPARK for fast analysis and processing of user requests. Server Agent and a Client Agent, each of which is deployed on the Server and Client side. The Server Agent is a necessary agent for the data provider and performs preliminary analysis of big data to generate Data Descriptor with information of Sample Data, Summary Data, and Raw Data. In addition, it performs fast and efficient big data preprocessing through big data distribution processing and continuously monitors network traffic. The Client Agent is an agent placed on the data user side. It can search the big data through the Data Descriptor which is the result of the pre-analysis and can quickly search the data. The desired data can be requested from the server to download the big data according to the level of disclosure. It separates the Server Agent and the client agent when the data provider publishes the data for data to be used by the user. In particular, we focus on the Big Data Sharing, Distributed Big Data Processing, Big Traffic problem, and construct the detailed module of the client - server model and present the design method of each module. The system designed on the basis of the proposed model, the user who acquires the data analyzes the data in the desired direction or preprocesses the new data. By analyzing the newly processed data through the server agent, the data user changes its role as the data provider. The data provider can also obtain useful statistical information from the Data Descriptor of the data it discloses and become a data user to perform new analysis using the sample data. In this way, raw data is processed and processed big data is utilized by the user, thereby forming a natural shared environment. The role of data provider and data user is not distinguished, and provides an ideal shared service that enables everyone to be a provider and a user. The client-server model solves the problem of sharing big data and provides a free sharing environment to securely big data disclosure and provides an ideal shared service to easily find big data.

Study on Educational Utilization Methods of Big Data (빅데이터의 교육적 활용 방안 연구)

  • Lee, Youngseok;Cho, Jungwon
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.17 no.12
    • /
    • pp.716-722
    • /
    • 2016
  • In the recent rapidly changing IT environment, the amount of smart digital data is growing exponentially. As a result, in many areas, utilizing big data research and development services and related technologies is becoming more popular. In SMART learning, big data is used by students, teachers, parents, etc., from a perspective of the potential for many. In this paper, we describe big data and can utilize it to identify scenarios. Big data, obtained through customized learning services that can take advantage of the scheme, is proposed. To analyze educational big data processing technology for this purpose, we designed a system for big data processing. Education services offer the measures necessary to take advantage of educational big data. These measures were implemented on a test platform that operates in a cloud-based operations section for a pilot training program that can be applied properly. Teachers try using it directly, and in the interest of business and education, a survey was conducted based on enjoyment, the tools, and users' feelings (e.g., tense, worried, confident). We analyzed the results to lay the groundwork for educational use of big data.

Big Data Model for Analyzing Plant Growth Environment Informations and Biometric Informations (농작물 생육환경정보와 생체정보 분석을 위한 빅데이터 모델)

  • Lee, JongYeol;Moon, ChangBae;Kim, ByeongMan
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.25 no.6
    • /
    • pp.15-23
    • /
    • 2020
  • While research activities in the agricultural field for climate change are being actively carried out, smart agriculture using information and communication technology has become a new trend in line with the Fourth Industrial Revolution. Accordingly, research is being conducted to identify and respond to signs of abnormal growth in advance by monitoring the stress of crops in various outdoor environments and soil conditions. There are also attempts to analyze data collected in real time through various sensors using artificial intelligence techniques or big data technologies. In this paper, we propose a big data model that is effective in analyzing the growth environment informations and biometric information of crops by using the existing relational database for big data analysis. The performance of the model was measured by the response time to a query according to the amount of data. As a result, it was confirmed that there is a maximum time reduction effect of 23.8%.