• 제목/요약/키워드: big data analysis

검색결과 3,341건 처리시간 0.038초

빅데이터 분석 방법을 이용한 중학교 정보 교과서 핵심 개념 분석 (Analysis of the Core Concepts of Middle School Informatics Textbook Using Big Data Analysis Techniques)

  • 윤대웅;최현종
    • 창의정보문화연구
    • /
    • 제5권2호
    • /
    • pp.157-164
    • /
    • 2019
  • 빅데이터는 최근 우리 사회에 다양한 분야에서 활용하고 발전하고 있는 분야이다. 정치, 경제, 사회의 각 분야에서 많은 자료를 분석하여 자료에 숨어 있는 다양한 의미를 파악하는 빅데이터 분석 기법이 자주 활용되고 있다. 하지만 교육이라는 부분에 있어서 빅데이터는 단순한 분석 등에 활용하기는 하지만 교육과정과 방향에 대한 분석은 아직 미비한 실정이다. 이에 본 연구에서는 빅데이터 분석 기법을 활용하여 중학교 정보 교과서의 핵심 개념을 파악하여 분석하고자 한다. 정보 교과서 분석을 위한 빅데이터 분석법은 텍스트 마이닝을 사용하였다. 이 프로그램을 활용하여 파악된 중학교 정보 교과서의 핵심 개념들을 통해 교과서에서 강조하고자 하는 개념을 확인할 수 있었고, 교육 분야에서도 빅데이터 활용 가능성을 확인할 수 있었다.

빅데이터와 통계학 (Big data and statistics)

  • 김용대;조광현
    • Journal of the Korean Data and Information Science Society
    • /
    • 제24권5호
    • /
    • pp.959-974
    • /
    • 2013
  • 빅데이터 시대를 맞이하여 통계학과 통계학자의 역할에 대하여 살펴본다. 빅데이터에 대한 정의 및 응용분야를 살펴보고, 빅데이터 자료의 통계학적 특징들 및 이와 관련한 통계학적 의의에 대해서 설명한다. 빅데이터 자료 분석에 유용하게 사용되는 통계적 방법론들에 대해서 살펴보고, 국외와 국내의 빅데이터 관련 프로젝트를 소개한다.

A Study on Policy and System Improvement Plan of Geo-Spatial Big Data Services in Korea

  • Park, Joon Min;Yu, Seon Cheol;Ahn, Jong Wook;Shin, Dong Bin
    • 한국측량학회지
    • /
    • 제34권6호
    • /
    • pp.579-589
    • /
    • 2016
  • This research focuses on accomplishing analysis problems and issues by examining the policies and systems related to geo-spatial big data which have recently arisen, and suggests political and systemic improvement plan for service activation. To do this, problems and probable issues concerning geo-spatial big data service activation should be analyzed through the examination of precedent studies, policies and planning, pilot projects, the current legislative situation regarding geo-spatial big data, both domestic and abroad. Therefore, eight political and systematical improvement plan proposals are suggested for geo-spatial big data service activation: legislative-related issues regarding geo-spatial big data, establishing an exclusive organization in charge of geospatial big data, setting up systems for cooperative governance, establishing subsequent systems, preparing non-identifying standards for personal information, providing measures for activating civil information, data standardization on geo-spatial big data analysis, developing analysis techniques for geo-spatial big data, etc. Consistent governmental problem-solving approaches should be required to make these suggestions effectively proceed.

빅 데이터의 새로운 고객 가치와 비즈니스 창출을 위한 대응 전략 (Correspondence Strategy for Big Data's New Customer Value and Creation of Business)

  • 고준철;이해욱;정지윤;강경식
    • 대한안전경영과학회지
    • /
    • 제14권4호
    • /
    • pp.229-238
    • /
    • 2012
  • Within last 10 years, internet has become a daily activity, and humankind had to face the Data Deluge, a dramatic increase of digital data (Economist 2012). Due to exponential increase in amount of digital data, large scale data has become a big issue and hence the term 'big data' appeared. There is no official agreement in quantitative and detailed definition of the 'big data', but the meaning is expanding to its value and efficacy. Big data not only has the standardized personal information (internal) like customer information, but also has complex data of external, atypical, social, and real time data. Big data's technology has the concept that covers wide range technology, including 'data achievement, save/manage, analysis, and application'. To define the connected technology of 'big data', there are Big Table, Cassandra, Hadoop, MapReduce, Hbase, and NoSQL, and for the sub-techniques, Text Mining, Opinion Mining, Social Network Analysis, Cluster Analysis are gaining attention. The three features that 'bid data' needs to have is about creating large amounts of individual elements (high-resolution) to variety of high-frequency data. Big data has three defining features of volume, variety, and velocity, which is called the '3V'. There is increase in complexity as the 4th feature, and as all 4features are satisfied, it becomes more suitable to a 'big data'. In this study, we have looked at various reasons why companies need to impose 'big data', ways of application, and advanced cases of domestic and foreign applications. To correspond effectively to 'big data' revolution, paradigm shift in areas of data production, distribution, and consumption is needed, and insight of unfolding and preparing future business by considering the unpredictable market of technology, industry environment, and flow of social demand is desperately needed.

Agriculture Big Data Analysis System Based on Korean Market Information

  • Chuluunsaikhan, Tserenpurev;Song, Jin-Hyun;Yoo, Kwan-Hee;Rah, Hyung-Chul;Nasridinov, Aziz
    • Journal of Multimedia Information System
    • /
    • 제6권4호
    • /
    • pp.217-224
    • /
    • 2019
  • As the world's population grows, how to maintain the food supply is becoming a bigger problem. Now and in the future, big data will play a major role in decision making in the agriculture industry. The challenge is how to obtain valuable information to help us make future decisions. Big data helps us to see history clearer, to obtain hidden values, and make the right decisions for the government and farmers. To contribute to solving this challenge, we developed the Agriculture Big Data Analysis System. The system consists of agricultural big data collection, big data analysis, and big data visualization. First, we collected structured data like price, climate, yield, etc., and unstructured data, such as news, blogs, TV programs, etc. Using the data that we collected, we implement prediction algorithms like ARIMA, Decision Tree, LDA, and LSTM to show the results in data visualizations.

Comparison of Sentiment Analysis from Large Twitter Datasets by Naïve Bayes and Natural Language Processing Methods

  • Back, Bong-Hyun;Ha, Il-Kyu
    • Journal of information and communication convergence engineering
    • /
    • 제17권4호
    • /
    • pp.239-245
    • /
    • 2019
  • Recently, effort to obtain various information from the vast amount of social network services (SNS) big data generated in daily life has expanded. SNS big data comprise sentences classified as unstructured data, which complicates data processing. As the amount of processing increases, a rapid processing technique is required to extract valuable information from SNS big data. We herein propose a system that can extract human sentiment information from vast amounts of SNS unstructured big data using the naïve Bayes algorithm and natural language processing (NLP). Furthermore, we analyze the effectiveness of the proposed method through various experiments. Based on sentiment accuracy analysis, experimental results showed that the machine learning method using the naïve Bayes algorithm afforded a 63.5% accuracy, which was lower than that yielded by the NLP method. However, based on data processing speed analysis, the machine learning method by the naïve Bayes algorithm demonstrated a processing performance that was approximately 5.4 times higher than that by the NLP method.

빅데이터 컴퓨팅을 위한 분석기법에 관한 연구 (A Study on the Analysis Techniques for Big Data Computing)

  • 오선진
    • 문화기술의 융합
    • /
    • 제7권3호
    • /
    • pp.475-480
    • /
    • 2021
  • 모바일 컴퓨팅과 클라우드 컴퓨팅 기술 그리고 소셜 네트워크 서비스의 급속한 발전과 더불어, 우리들은 시시각각 양산되고 있는 데이터의 홍수 속에서 살고 있으며, 이러한 대규모의 데이터는 매우 가치가 높은 중요한 정보를 품고 있다는 사실을 알게 되었다. 하지만 빅데이터는 잠재적인 유용한 가치와 치명적인 위험을 모두 가지고 있으며 오늘날 이러한 빅데이터로부터 유용한 정보를 효율적으로 추출해 내고 잠재된 정보를 효과적으로 활용하기 위한 연구와 응용이 활발하게 이루어지고 있는 상황이다. 여기서 빅데이터 컴퓨팅 과정 중 무엇보다도 중요한 것은 대용량 데이터로부터 유용하고 귀중한 정보를 효율적으로 추출해 낼 수 있는 적절한 데이터 분석기법을 찾아 적용하는 것이다. 본 연구에서는 이러한 빅데이터 컴퓨팅을 효율적으로 수행하여 원하는 유용한 정보를 추출할 수 있는 기존의 다양한 빅데이터 분석기법들을 조사하여, 그 특징과 장·단점 등을 비교 분석하고, 특별한 상황에서 빅데이터 분석기법을 이용하여 유용한 정보를 효율적으로 추출해 내고, 이들 잠재된 정보를 효과적으로 활용할 수 있도록 하는 방안을 제시하고자 한다.

로그 분석 처리율 향상을 위한 맵리듀스 기반 분할 빅데이터 분석 기법 (MapReduce-Based Partitioner Big Data Analysis Scheme for Processing Rate of Log Analysis)

  • 이협건;김영운;박지용;이진우
    • 한국정보전자통신기술학회논문지
    • /
    • 제11권5호
    • /
    • pp.593-600
    • /
    • 2018
  • 인터넷과 스마트기기의 발달로 인해 소셜미디어 등 다양한 미디어의 접근의 용이해짐에 따라 많은 양의 빅데이터들이 생성되고 있다. 특히 다양한 인터넷 서비스를 제공하는 기업들은 고객 성향 및 패턴, 보안성 강화를 위해 맵리듀스 기반 빅데이터 분석 기법들을 활용하여 빅데이터 분석하고 있다. 그러나 맵리듀스는 리듀스 단계에서 생성되는 리듀서 객체의 수를 한 개로 정의하고 있어, 빅데이터 분석할 때 처리될 많은 데이터들이 하나의 리듀서 객체에 집중된다. 이로 인해 리듀서 객체는 병목현상이 발생으로 빅데이터 분석 처리율이 감소한다. 이에 본 논문에서는 로그 분석처리율 향상을 위한 맵리듀스 기반 분할 빅데이터 분석 기법을 제안한다. 제안한 기법은 리듀서 분할 단계와 분석 결과병합 단계로 구분하며 리듀서 객체의 수를 유동적으로 생성하여 병목현상을 감소시켜 빅데이터 처리율을 향상시킨다.

데이터 사이언티스트의 역량과 빅데이터 분석성과의 PLS 경로모형분석 : Kaggle 플랫폼을 중심으로 (PLS Path Modeling to Investigate the Relations between Competencies of Data Scientist and Big Data Analysis Performance : Focused on Kaggle Platform)

  • 한경진;조근태
    • 대한산업공학회지
    • /
    • 제42권2호
    • /
    • pp.112-121
    • /
    • 2016
  • This paper focuses on competencies of data scientists and behavioral intention that affect big data analysis performance. This experiment examined nine core factors required by data scientists. In order to investigate this, we conducted a survey to gather data from 103 data scientists who participated in big data competition at Kaggle platform and used factor analysis and PLS-SEM for the analysis methods. The results show that some key competency factors have influential effect on the big data analysis performance. This study is to provide a new theoretical basis needed for relevant research by analyzing the structural relationship between the individual competencies and performance, and practically to identify the priorities of the core competencies that data scientists must have.

A Study on the Sentiment Analysis of City Tour Using Big Data

  • Se-won Jeon;Gi-Hwan Ryu
    • International Journal of Internet, Broadcasting and Communication
    • /
    • 제15권2호
    • /
    • pp.112-117
    • /
    • 2023
  • This study aims to find out what tourists' interests and perceptions are like through online big data. Big data for a total of five years from 2018 to 2022 were collected using the Textom program. Sentiment analysis was performed with the collected data. Sentiment analysis expresses the necessity and emotions of city tours in online reviews written by tourists using city tours. The purpose of this study is to extract and analyze keywords representing satisfaction. The sentiment analysis program provided by the big data analysis platform "TEXTOM" was used to study positives and negatives based on sentiment analysis of tourists' online reviews. Sentiment analysis was conducted by collecting reviews related to the city tour. The degree of positive and negative emotions for the city tour was investigated and what emotional words were analyzed for each item. As a result of big data sentiment analysis to examine the emotions and sentiments of tourists about the city tour, 93.8% positive and 6.2% negative, indicating that more than half of the tourists are positively aware. This paper collects tourists' opinions based on the analyzed sentiment analysis, understands the quality characteristics of city tours based on the analysis using the collected data, and sentiment analysis provides important information to the city tour platform for each region.