• Title/Summary/Keyword: big data analysis

Search Result 3,427, Processing Time 0.033 seconds

Analysis of the Core Concepts of Middle School Informatics Textbook Using Big Data Analysis Techniques (빅데이터 분석 방법을 이용한 중학교 정보 교과서 핵심 개념 분석)

  • Woon, Daewoong;Choe, Hyunjong
    • Journal of Creative Information Culture
    • /
    • v.5 no.2
    • /
    • pp.157-164
    • /
    • 2019
  • Big data is a field that has been utilized and developed in various fields in our society recently. Big data analysis techniques are frequently used to analyze various big data in various fields of politics, economy, and society to grasp various meanings hidden in the data. However, big data analysis is used some case studies of in fields of analysis of educational data, but analysis of the curriculum and direction is still inadequate. Therefore, this study aims to identify and analyze the core concepts of middle school informatics textbooks using big data analysis techniques. Text mining was used for big data analysis for informatics textbook analysis. Through the core concepts of middle school informatics textbooks identified using this techniques, we could confirm the concepts to be emphasized in the textbooks and the possibility of using big data in the field of education.

Big data and statistics (빅데이터와 통계학)

  • Kim, Yongdai;Cho, Kwang Hyun
    • Journal of the Korean Data and Information Science Society
    • /
    • v.24 no.5
    • /
    • pp.959-974
    • /
    • 2013
  • We investigate the roles of statistics and statisticians in the big data era. Definition and application areas of big data are reviewed and statistical characteristics of big data and their meanings are discussed. Various statistical methodologies applicable to big data analysis are illustrated, and two real big data projects are explained.

A Study on Policy and System Improvement Plan of Geo-Spatial Big Data Services in Korea

  • Park, Joon Min;Yu, Seon Cheol;Ahn, Jong Wook;Shin, Dong Bin
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.34 no.6
    • /
    • pp.579-589
    • /
    • 2016
  • This research focuses on accomplishing analysis problems and issues by examining the policies and systems related to geo-spatial big data which have recently arisen, and suggests political and systemic improvement plan for service activation. To do this, problems and probable issues concerning geo-spatial big data service activation should be analyzed through the examination of precedent studies, policies and planning, pilot projects, the current legislative situation regarding geo-spatial big data, both domestic and abroad. Therefore, eight political and systematical improvement plan proposals are suggested for geo-spatial big data service activation: legislative-related issues regarding geo-spatial big data, establishing an exclusive organization in charge of geospatial big data, setting up systems for cooperative governance, establishing subsequent systems, preparing non-identifying standards for personal information, providing measures for activating civil information, data standardization on geo-spatial big data analysis, developing analysis techniques for geo-spatial big data, etc. Consistent governmental problem-solving approaches should be required to make these suggestions effectively proceed.

Correspondence Strategy for Big Data's New Customer Value and Creation of Business (빅 데이터의 새로운 고객 가치와 비즈니스 창출을 위한 대응 전략)

  • Koh, Joon-Cheol;Lee, Hae-Uk;Jeong, Jee-Youn;Kim, Kyung-Sik
    • Journal of the Korea Safety Management & Science
    • /
    • v.14 no.4
    • /
    • pp.229-238
    • /
    • 2012
  • Within last 10 years, internet has become a daily activity, and humankind had to face the Data Deluge, a dramatic increase of digital data (Economist 2012). Due to exponential increase in amount of digital data, large scale data has become a big issue and hence the term 'big data' appeared. There is no official agreement in quantitative and detailed definition of the 'big data', but the meaning is expanding to its value and efficacy. Big data not only has the standardized personal information (internal) like customer information, but also has complex data of external, atypical, social, and real time data. Big data's technology has the concept that covers wide range technology, including 'data achievement, save/manage, analysis, and application'. To define the connected technology of 'big data', there are Big Table, Cassandra, Hadoop, MapReduce, Hbase, and NoSQL, and for the sub-techniques, Text Mining, Opinion Mining, Social Network Analysis, Cluster Analysis are gaining attention. The three features that 'bid data' needs to have is about creating large amounts of individual elements (high-resolution) to variety of high-frequency data. Big data has three defining features of volume, variety, and velocity, which is called the '3V'. There is increase in complexity as the 4th feature, and as all 4features are satisfied, it becomes more suitable to a 'big data'. In this study, we have looked at various reasons why companies need to impose 'big data', ways of application, and advanced cases of domestic and foreign applications. To correspond effectively to 'big data' revolution, paradigm shift in areas of data production, distribution, and consumption is needed, and insight of unfolding and preparing future business by considering the unpredictable market of technology, industry environment, and flow of social demand is desperately needed.

Agriculture Big Data Analysis System Based on Korean Market Information

  • Chuluunsaikhan, Tserenpurev;Song, Jin-Hyun;Yoo, Kwan-Hee;Rah, Hyung-Chul;Nasridinov, Aziz
    • Journal of Multimedia Information System
    • /
    • v.6 no.4
    • /
    • pp.217-224
    • /
    • 2019
  • As the world's population grows, how to maintain the food supply is becoming a bigger problem. Now and in the future, big data will play a major role in decision making in the agriculture industry. The challenge is how to obtain valuable information to help us make future decisions. Big data helps us to see history clearer, to obtain hidden values, and make the right decisions for the government and farmers. To contribute to solving this challenge, we developed the Agriculture Big Data Analysis System. The system consists of agricultural big data collection, big data analysis, and big data visualization. First, we collected structured data like price, climate, yield, etc., and unstructured data, such as news, blogs, TV programs, etc. Using the data that we collected, we implement prediction algorithms like ARIMA, Decision Tree, LDA, and LSTM to show the results in data visualizations.

Comparison of Sentiment Analysis from Large Twitter Datasets by Naïve Bayes and Natural Language Processing Methods

  • Back, Bong-Hyun;Ha, Il-Kyu
    • Journal of information and communication convergence engineering
    • /
    • v.17 no.4
    • /
    • pp.239-245
    • /
    • 2019
  • Recently, effort to obtain various information from the vast amount of social network services (SNS) big data generated in daily life has expanded. SNS big data comprise sentences classified as unstructured data, which complicates data processing. As the amount of processing increases, a rapid processing technique is required to extract valuable information from SNS big data. We herein propose a system that can extract human sentiment information from vast amounts of SNS unstructured big data using the naïve Bayes algorithm and natural language processing (NLP). Furthermore, we analyze the effectiveness of the proposed method through various experiments. Based on sentiment accuracy analysis, experimental results showed that the machine learning method using the naïve Bayes algorithm afforded a 63.5% accuracy, which was lower than that yielded by the NLP method. However, based on data processing speed analysis, the machine learning method by the naïve Bayes algorithm demonstrated a processing performance that was approximately 5.4 times higher than that by the NLP method.

A Study on the Analysis Techniques for Big Data Computing (빅데이터 컴퓨팅을 위한 분석기법에 관한 연구)

  • Oh, Sun-Jin
    • The Journal of the Convergence on Culture Technology
    • /
    • v.7 no.3
    • /
    • pp.475-480
    • /
    • 2021
  • With the rapid development of mobile, cloud computing technology and social network services, we are in the flood of huge data and realize that these large-scale data contain very precious value and important information. Big data, however, have both latent useful value and critical risks, so, nowadays, a lot of researches and applications for big data has been executed actively in order to extract useful information from big data efficiently and make the most of the potential information effectively. At this moment, the data analysis technique that can extract precious information from big data efficiently is the most important step in big data computing process. In this study, we investigate various data analysis techniques that can extract the most useful information in big data computing process efficiently, compare pros and cons of those techniques, and propose proper data analysis method that can help us to find out the best solution of the big data analysis in the peculiar situation.

MapReduce-Based Partitioner Big Data Analysis Scheme for Processing Rate of Log Analysis (로그 분석 처리율 향상을 위한 맵리듀스 기반 분할 빅데이터 분석 기법)

  • Lee, Hyeopgeon;Kim, Young-Woon;Park, Jiyong;Lee, Jin-Woo
    • The Journal of Korea Institute of Information, Electronics, and Communication Technology
    • /
    • v.11 no.5
    • /
    • pp.593-600
    • /
    • 2018
  • Owing to the advancement of Internet and smart devices, access to various media such as social media became easy; thus, a large amount of big data is being produced. Particularly, the companies that provide various Internet services are analyzing the big data by using the MapReduce-based big data analysis techniques to investigate the customer preferences and patterns and strengthen the security. However, with MapReduce, when the big data is analyzed by defining the number of reducer objects generated in the reduce stage as one, the processing rate of big data analysis decreases. Therefore, in this paper, a MapReduce-based split big data analysis method is proposed to improve the log analysis processing rate. The proposed method separates the reducer partitioning stage and the analysis result combining stage and improves the big data processing rate by decreasing the bottleneck phenomenon by generating the number of reducer objects dynamically.

PLS Path Modeling to Investigate the Relations between Competencies of Data Scientist and Big Data Analysis Performance : Focused on Kaggle Platform (데이터 사이언티스트의 역량과 빅데이터 분석성과의 PLS 경로모형분석 : Kaggle 플랫폼을 중심으로)

  • Han, Gyeong Jin;Cho, Keuntae
    • Journal of Korean Institute of Industrial Engineers
    • /
    • v.42 no.2
    • /
    • pp.112-121
    • /
    • 2016
  • This paper focuses on competencies of data scientists and behavioral intention that affect big data analysis performance. This experiment examined nine core factors required by data scientists. In order to investigate this, we conducted a survey to gather data from 103 data scientists who participated in big data competition at Kaggle platform and used factor analysis and PLS-SEM for the analysis methods. The results show that some key competency factors have influential effect on the big data analysis performance. This study is to provide a new theoretical basis needed for relevant research by analyzing the structural relationship between the individual competencies and performance, and practically to identify the priorities of the core competencies that data scientists must have.

A Study on the Sentiment Analysis of City Tour Using Big Data

  • Se-won Jeon;Gi-Hwan Ryu
    • International Journal of Internet, Broadcasting and Communication
    • /
    • v.15 no.2
    • /
    • pp.112-117
    • /
    • 2023
  • This study aims to find out what tourists' interests and perceptions are like through online big data. Big data for a total of five years from 2018 to 2022 were collected using the Textom program. Sentiment analysis was performed with the collected data. Sentiment analysis expresses the necessity and emotions of city tours in online reviews written by tourists using city tours. The purpose of this study is to extract and analyze keywords representing satisfaction. The sentiment analysis program provided by the big data analysis platform "TEXTOM" was used to study positives and negatives based on sentiment analysis of tourists' online reviews. Sentiment analysis was conducted by collecting reviews related to the city tour. The degree of positive and negative emotions for the city tour was investigated and what emotional words were analyzed for each item. As a result of big data sentiment analysis to examine the emotions and sentiments of tourists about the city tour, 93.8% positive and 6.2% negative, indicating that more than half of the tourists are positively aware. This paper collects tourists' opinions based on the analyzed sentiment analysis, understands the quality characteristics of city tours based on the analysis using the collected data, and sentiment analysis provides important information to the city tour platform for each region.