• 제목/요약/키워드: big data analysis

검색결과 3,341건 처리시간 0.034초

하둡과 순차패턴 마이닝 기술을 통한 교통카드 빅데이터 분석 (Analysis of Traffic Card Big Data by Hadoop and Sequential Mining Technique)

  • 김우생;김용훈;박희성;박진규
    • Journal of Information Technology Applications and Management
    • /
    • 제24권4호
    • /
    • pp.187-196
    • /
    • 2017
  • It is urgent to prepare countermeasures for traffic congestion problems of Korea's metropolitan area where central functions such as economic, social, cultural, and education are excessively concentrated. Most users of public transportation in metropolitan areas including Seoul use the traffic cards. If various information is extracted from traffic big data produced by the traffic cards, they can provide basic data for transport policies, land usages, or facility plans. Therefore, in this study, we extract valuable information such as the subway passengers' frequent travel patterns from the big traffic data provided by the Seoul Metropolitan Government Big Data Campus. For this, we use a Hadoop (High-Availability Distributed Object-Oriented Platform) to preprocess the big data and store it into a Mongo database in order to analyze it by a sequential pattern data mining technique. Since we analysis the actual big data, that is, the traffic cards' data provided by the Seoul Metropolitan Government Big Data Campus, the analyzed results can be used as an important referenced data when the Seoul government makes a plan about the metropolitan traffic policies.

DTG Big Data Analysis for Fuel Consumption Estimation

  • Cho, Wonhee;Choi, Eunmi
    • Journal of Information Processing Systems
    • /
    • 제13권2호
    • /
    • pp.285-304
    • /
    • 2017
  • Big data information and pattern analysis have applications in many industrial sectors. To reduce energy consumption effectively, the eco-driving method that reduces the fuel consumption of vehicles has recently come under scrutiny. Using big data on commercial vehicles obtained from digital tachographs (DTGs), it is possible not only to aid traffic safety but also improve eco-driving. In this study, we estimate fuel consumption efficiency by processing and analyzing DTG big data for commercial vehicles using parallel processing with the MapReduce mechanism. Compared to the conventional measurement of fuel consumption using the On-Board Diagnostics II (OBD-II) device, in this paper, we use actual DTG data and OBD-II fuel consumption data to identify meaningful relationships to calculate fuel efficiency rates. Based on the driving pattern extracted from DTG data, estimating fuel consumption is possible by analyzing driving patterns obtained only from DTG big data.

4차 산업혁명 시대에 적합한 빅데이터 대학 교육과정 연구 (Research on big data curriculum in university suitable for the era of the 4th industrial revolution)

  • Choi, Hun;Kim, Gimun
    • 한국정보통신학회논문지
    • /
    • 제24권11호
    • /
    • pp.1562-1565
    • /
    • 2020
  • With the development of digital technology, the industrial structure is becoming digitalize. The government selected big data as the key technology of the 4th industrial revolution. Among them, big data is widely used to create new values and services by utilizing vast amounts of information. In order to cultivate professional manpower for the use of big data, various education programs are provided at universities. We intend to develop a curriculum for systematic training of talented people who can acquire knowledge about the three stages of collection, analysis, and application of big data. To this end, subjects are classified into basic competency, technical competency, analysis competency, and business competency based on the big data competency model proposed by the Korea Internet & Security Agency.

SWOT분석을 통한 CM사 견적업무 빅데이터 활용전략에 관한 연구 (A Study on the Strategy of the Use of Big Data for Cost Estimating in Construction Management Firms based on the SWOT Analysis)

  • 김현진;김한수
    • 한국건설관리학회논문집
    • /
    • 제23권2호
    • /
    • pp.54-64
    • /
    • 2022
  • 빅데이터 활용에 대한 관심이 높아짐에 따라, 건설산업에서도 빅데이터와 관련한 다양한 연구개발이 이루어지고 있다. 건설산업의 다양한 분야 중 견적업무는 빅데이터의 활용성이 높은 분야로 인식되고 있다. 견적업무에서 빅데이터를 효과적으로 활용하기 위해서는, 기업의 내외부 현황을 다면적으로 이해하고 이에 적합한 활용전략을 수립하는 것이 필요할 것이다. 본 연구의 목적은 국내 CM사 견적업무에서의 빅데이터 활용현황을 조사하고, SWOT기법을 활용하여 CM사 견적업무에서 빅데이터를 활용하기 위한 전략 방향을 개발하고 제시하는데 있다. 문헌조사, 설문조사, 인터뷰 조사 및 SWOT분석을 바탕으로 CM사는 기업의 높은 수용 문화와 정보 자원을 적극 활용하고, 부족한 빅데이터 실무기반과 인적자원을 보강하는 전략이 필요한 것으로 제안하였다.

A Trend Analysis of Floral Products and Services Using Big Data of Social Networking Services

  • Park, Sin Young;Oh, Wook
    • 인간식물환경학회지
    • /
    • 제22권5호
    • /
    • pp.455-466
    • /
    • 2019
  • This study was carried out to analyze trends in floral products and services through the big data analysis of various social networking services (SNSs) and then to provide objective marketing directions for the floricultural industry. To analyze the big data of SNSs, we used four analytical methods: Cotton Trend (Social Matrix), Naver Big Data Lab, Instagram Big Data Analysis, and YouTube Big Data Analysis. The results of the big data analysis showed that SNS users paid positive attention to flower one-day classes that can satisfy their needs for direct experiences. Consumers of floral products and services had their favorite designs in mind and purchased floral products very actively. The demand for flower items such as bouquets, wreaths, flower baskets, large bouquets, orchids, flower boxes, wedding bouquets, and potted plants was very high, and cut flowers such as roses, tulips, and freesia were most popular as of June 1, 2019. By gender of consumers, females (68%) purchased more flower products through SNSs than males (32%). Consumers preferred mobile devices (90%) for online access compared to personal computers (PCs; 10%) and frequently searched flower-related words from February to May for the past three years from 2016 to 2018. In the aspect of design, they preferred natural style to formal style. In conclusion, future marketing activities in the floricultural industry need to be focused on social networks based on the results of big data analysis of popular SNSs. Florists need to provide consumers with the floricultural products and services that meet the trends and to blend them with their own sensitivity. It is also needed to select SNS media suitable for each gender and age group and to apply effective marketing methods to each target.

Big Data Platform Based on Hadoop and Application to Weight Estimation of FPSO Topside

  • Kim, Seong-Hoon;Roh, Myung-Il;Kim, Ki-Su;Oh, Min-Jae
    • Journal of Advanced Research in Ocean Engineering
    • /
    • 제3권1호
    • /
    • pp.32-40
    • /
    • 2017
  • Recently, the amount of data to be processed and the complexity thereof have been increasing due to the development of information and communication technology, and industry's interest in such big data is increasing day by day. In the shipbuilding and offshore industry also, there is growing interest in the effective utilization of data, since various and vast amounts of data are being generated in the process of design, production, and operation. In order to effectively utilize big data in the shipbuilding and offshore industry, it is necessary to store and process large amounts of data. In this study, it was considered efficient to apply Hadoop and R, which are mostly used in big data related research. Hadoop is a framework for storing and processing big data. It provides the Hadoop Distributed File System (HDFS) for storing big data, and the MapReduce function for processing. Meanwhile, R provides various data analysis techniques through the language and environment for statistical calculation and graphics. While Hadoop makes it is easy to handle big data, it is difficult to finely process data; and although R has advanced analysis capability, it is difficult to use to process large data. This study proposes a big data platform based on Hadoop for applications in the shipbuilding and offshore industry. The proposed platform includes the existing data of the shipyard, and makes it possible to manage and process the data. To check the applicability of the platform, it is applied to estimate the weights of offshore structure topsides. In this study, we store data of existing FPSOs in Hadoop-based Hortonworks Data Platform (HDP), and perform regression analysis using RHadoop. We evaluate the effectiveness of large data processing by RHadoop by comparing the results of regression analysis and the processing time, with the results of using the conventional weight estimation program.

A Study on the Classification of Variables Affecting Smartphone Addiction in Decision Tree Environment Using Python Program

  • Kim, Seung-Jae
    • International journal of advanced smart convergence
    • /
    • 제11권4호
    • /
    • pp.68-80
    • /
    • 2022
  • Since the launch of AI, technology development to implement complete and sophisticated AI functions has continued. In efforts to develop technologies for complete automation, Machine Learning techniques and deep learning techniques are mainly used. These techniques deal with supervised learning, unsupervised learning, and reinforcement learning as internal technical elements, and use the Big-data Analysis method again to set the cornerstone for decision-making. In addition, established decision-making is being improved through subsequent repetition and renewal of decision-making standards. In other words, big data analysis, which enables data classification and recognition/recognition, is important enough to be called a key technical element of AI function. Therefore, big data analysis itself is important and requires sophisticated analysis. In this study, among various tools that can analyze big data, we will use a Python program to find out what variables can affect addiction according to smartphone use in a decision tree environment. We the Python program checks whether data classification by decision tree shows the same performance as other tools, and sees if it can give reliability to decision-making about the addictiveness of smartphone use. Through the results of this study, it can be seen that there is no problem in performing big data analysis using any of the various statistical tools such as Python and R when analyzing big data.

Design and Development of Big Data Platform based on IoT-based Children's Play Pattern Analysis

  • Jung, Seon-Jin
    • International Journal of Internet, Broadcasting and Communication
    • /
    • 제12권4호
    • /
    • pp.218-225
    • /
    • 2020
  • The purpose of this paper is to establish an IoT-based big data platform that can check the space and form analysis in various play cultures of children. Therefore, to this end, in order to understand the healthy play culture of children, we are going to build a big data platform that allows IoT and smart devices to work together to collect data. Therefore, the goal of this study is to develop a big data platform linked to IoT first in order to collect data related to observation of children's mobile movements. Using the developed big data platform, children's play culture can be checked anywhere through observation and intuitive UI design, quick information can be automatically collected and real-time feedback, data collected through repeaters can be aggregated and analyzed, and systematic database can be utilized in the form of big data.

Big data-based piping material analysis framework in offshore structure for contract design

  • Oh, Min-Jae;Roh, Myung-Il;Park, Sung-Woo;Chun, Do-Hyun;Myung, Sehyun
    • Ocean Systems Engineering
    • /
    • 제9권1호
    • /
    • pp.79-95
    • /
    • 2019
  • The material analysis of an offshore structure is generally conducted in the contract design phase for the price quotation of a new offshore project. This analysis is conducted manually by an engineer, which is time-consuming and can lead to inaccurate results, because the data size from previous projects is too large, and there are so many materials to consider. In this study, the piping materials in an offshore structure are analyzed for contract design using a big data framework. The big data technologies used include HDFS (Hadoop Distributed File System) for data saving, Hive and HBase for the database to handle the saved data, Spark and Kylin for data processing, and Zeppelin for user interface and visualization. The analyzed results show that the proposed big data framework can reduce the efforts put toward contract design in the estimation of the piping material cost.

빅데이터 분석 도구 R을 이용한 비정형 데이터 텍스트 마이닝과 시각화 (Text Mining and Visualization of Unstructured Data Using Big Data Analytical Tool R)

  • 남수태;신성윤;진찬용
    • 한국정보통신학회논문지
    • /
    • 제25권9호
    • /
    • pp.1199-1205
    • /
    • 2021
  • 빅데이터 시대에는 단순히 데이터베이스에 잘 정리된 정형 데이터뿐만 아니라 인터넷, 소셜 네트워크 서비스, 모바일 환경에서 실시간 생성되는 웹 문서, 이메일, 소셜 데이터 등 비정형 빅데이터를 효과적으로 분석하는 것이 매우 중요하다. 빅데이터 분석은 데이터 저장소에 저장된 빅데이터 속에서 의미 있는 새로운 상관관계, 패턴, 추세를 발견하여 새로운 가치를 창출하는 과정이다. 빅데이터 분석 도구인 R 언어를 이용하여 비정형 논문 데이터를 빈도분석을 통해 분석결과를 요약과 시각화하고자 한다. 본 연구에서 사용된 데이터는 한국정보통신학회 학회지 논문 중에서 2021년 1월호-5월호 총 논문 104편을 대상으로 분석하였다. 최종 분석결과 가장 많이 언급된 키워드는 "데이터"가 1,538회로 1위를 차지하였다. 따라서 분석결과를 바탕으로 연구의 한계와 이론적 실무적 시사점을 제시하고자 한다.