• Title/Summary/Keyword: bigdata analysis

Search Result 345, Processing Time 0.03 seconds

Infrastructure Anomaly Analysis for Data-center Failure Prevention: Based on RRCF and Prophet Ensemble Analysis (데이터센터 장애 예방을 위한 인프라 이상징후 분석: RRCF와 Prophet Ensemble 분석 기반)

  • Hyun-Jong Kim;Sung-Keun Kim;Byoung-Whan Chun;Kyong-Bog, Jin;Seung-Jeong Yang
    • The Journal of Bigdata
    • /
    • v.7 no.1
    • /
    • pp.113-124
    • /
    • 2022
  • Various methods using machine learning and big data have been applied to prevent failures in Data Centers. However, there are many limitations to referencing individual equipment-based performance indicators or to being practically utilized as an approach that does not consider the infrastructure operating environment. In this study, the performance indicators of individual infrastructure equipment are integrated monitoring and the performance indicators of various equipment are segmented and graded to make a single numerical value. Data pre-processing based on experience in infrastructure operation. And an ensemble of RRCF (Robust Random Cut Forest) analysis and Prophet analysis model led to reliable analysis results in detecting anomalies. A failure analysis system was implemented to facilitate the use of Data Center operators. It can provide a preemptive response to Data Center failures and an appropriate tuning time.

The Analysis Framework for User Behavior Model using Massive Transaction Log Data (대규모 로그를 사용한 유저 행동모델 분석 방법론)

  • Lee, Jongseo;Kim, Songkuk
    • The Journal of Bigdata
    • /
    • v.1 no.2
    • /
    • pp.1-8
    • /
    • 2016
  • User activity log includes lots of hidden information, however it is not structured and too massive to process data, so there are lots of parts uncovered yet. Especially, it includes time series data. We can reveal lots of parts using it. But we cannot use log data directly to analyze users' behaviors. In order to analyze user activity model, it needs transformation process through extra framework. Due to these things, we need to figure out user activity model analysis framework first and access to data. In this paper, we suggest a novel framework model in order to analyze user activity model effectively. This model includes MapReduce process for analyzing massive data quickly in the distributed environment and data architecture design for analyzing user activity model. Also we explained data model in detail based on real online service log design. Through this process, we describe which analysis model is fit for specific data model. It raises understanding of processing massive log and designing analysis model.

  • PDF

A Study on Tourism Resource Strategy of Film Location using Social Bigdata based on SNS Trend Analysis of Jeonju Area (소셜 빅데이터를 활용한 영화촬영지 관광자원화 방안 -전주 지역의 관광체험 SNS 동향 분석을 토대로-)

  • Park, Ji-Yeong;Kim, Geon;Kim, Chan-Young;Oh, Hyo-Jung
    • The Journal of the Korea Contents Association
    • /
    • v.16 no.11
    • /
    • pp.477-487
    • /
    • 2016
  • In 1995, the filming location of the drama had been famous, and as a result it brings the effect of increasing tourists of that areas. After that, many local governments try to host the filming on their regions to be potential tourist attractions. With the same stream, Jeonju also has attempted to host International Film Festival and to set up Jeonju Film Commission and Jeonju Cinema Complex. However, although the city already has rich infrastructure facilities to make films, the city hardly tries to use the filming locations as tourist attractions. This study suggests four ways of using filming locations as tourist attractions to activate Jeonju economy and improve Jeonju's cultural image. We firstly collect social bigdata related with tourists of filming locations and tourist attractions in Jeonju from Twitter, which is the most representative SNS, and then perform frequency and trend analysis. We also investigate major factors of visits to tourist's attractions based on content analysis of tweet mentions.

The Comparison Among Prediction Methods of Water Demand And Analysis of Data on Water Services Using Data Mining Techniques (데이터마이닝 기법을 활용한 상수 이용현황 분석 및 단기 물 수요예측 방법 비교)

  • Ahn, Jihoon;Kim, Jinhwa
    • The Journal of Bigdata
    • /
    • v.1 no.1
    • /
    • pp.9-17
    • /
    • 2016
  • This study identifies major features in water supply and introduces important factors in water services based on the information from data mining analysis of water quantity and water pressure measured from sensors. It also suggests more accurate methods using multiple regression analysis and neural network in predicting short term prediction of water demand in water service. A small block of a county is selected for the data collection and tests. There isa water demand on business such as public offices and hospitalstoo in this area. Real stream data from sensors in this area is collected. Among 2,728 data sets collected, 2,632 sets are used for modelling and 96 sets are used for testing. The shows that neural network is better than multiple regression analysis in their prediction performance.

  • PDF

Big data Cloud Service for Manufacturing Process Analysis (제조 공정 분석을 위한 빅데이터 클라우드 서비스)

  • Lee, Yong-Hyeok;Song, Min-Seok;Ha, Seung-Jin;Baek, Tae-Hyun;Son, Sook-Young
    • The Journal of Bigdata
    • /
    • v.1 no.1
    • /
    • pp.41-51
    • /
    • 2016
  • Big data is an emerging issue as large data which was impossible to be processed in the past is possible to be handled with the development of information and communication technology. Manufacturing is the most promising field that big data is applied such that there are abundant data available. It is important to improve an efficiency of manufacturing process for quality control and production efficiency because the processes from production design, sales, productions and so on are mixed intricately. This study proposes big data cloud service for manufacturing analysis using a big data technology and a process mining technique. It is expected for manufacturing corporations to improve a manufacturing process and reduced the cost by applying the proposed service. The service provides various analyses including manufacturing analysis and manufacturing duration analysis. Big data cloud service has been implemented and it has been validated by conducting a case study.

  • PDF

A Researh for Consumer Dissatisfaction and Institutional Improvement of The Overseas Direct Purchase using Exploratory Data Analysis (탐색적 자료 분석(EDA) 기법을 활용한 온라인 해외직접구매에 대한 소비자 불만족 및 제도 개선 방안 연구)

  • Park, Seongwoo;Kang, Juyoung
    • The Journal of Bigdata
    • /
    • v.5 no.1
    • /
    • pp.41-54
    • /
    • 2020
  • With the recent expansion of Internet channels and the development of financial technology and information and communication technology, direct overseas purchases have expanded. Although direct overseas purchases dominate consumers in terms of price and scarcity by providing relatively low-priced products and products that are difficult to obtain in Korea, there is a higher chance of consumer dissatisfaction in terms of delivery, product, A/S and refund than domestic purchases. Therefore, this study analyzed consumer dissatisfaction caused by active overseas direct purchase and studied ways to improve problems with overseas direct purchase. As a research method, Several statistical data were collected from the Korea Consumer Agency(KCA), the Korea Customs Service(KCS) and the Korea International Trade Association(KITA) and analyzed using the Exploratory Data Analysis Technique (EDA). The analysis confirmed that consumers were not well aware of information about direct overseas purchases and that the type or degree of consumer complaints varied depending on the type of purchase. Therefore, this study suggests a direction for the revitalization of overseas direct purchases by using EDA to identify the overall status of overseas direct purchases and consumer dissatisfaction and to improve them.

A Network Analysis on Industry-University Cooperation based on Big Data Analytics (빅데이터 기반 산학협력 네트워크 분석)

  • Dae-Hee Kang;Hyunchul Ahn
    • The Journal of Bigdata
    • /
    • v.6 no.2
    • /
    • pp.109-124
    • /
    • 2021
  • In this paper, the structural characteristics of Industry-University cooperation networks are analyzed using network analysis. Recent studies have shown that technological cooperation and joint research has a positive effect on R&D performance. In order to boost innovation performance, various types of cooperative activities and governmental policy supports for major R&D stakeholders(i.e. universities, laboratories, etc.) are provided. However, despite these efforts, the outcome is still insufficient, so it is time to prepare for a plan to build an innovative network to strengthen university-centered Industry-University cooperation activities. Specifically, this study builds the networks according to the form of Industry-University cooperations(i.e. patent, paper, joint research, and technology transfer), and different types of Industry-University cooperation networks are analyzed from a statistical viewpoint by using QAP correlation and regression analyses. The analysis results show that joint research network is closely related to paper network, and is related to other Industry-University cooperation networks. This study is expected to shed a light on supporting innovation activities such as establishing Industry-University cooperation strategies and discovering cooperative partners necessary for creating new growth engines for universities.

Bigdata Analysis Project Development Methodology (빅데이터 분석 프로젝트 수행 방법론)

  • Kim, Hyoungrae;Jeon, Do-hong;Jee, Sunghyun
    • Journal of the Korea Society of Computer and Information
    • /
    • v.19 no.3
    • /
    • pp.73-85
    • /
    • 2014
  • As the importance of big data analysis increases to improve the competitiveness of a corporate, a unified big data project development methodology is required in order to study the problem of a corporate in a systematic way and evaluate the problem w.r.t. a business value after solving the problem. This paper propose Scientific Data Anslysis and Development methodology(SDAD) which are integrated methodology of software development and project management for easier application into a field project. SDAD consisits of 6 stages(problem definition stage, data preparation stage, model design stage, model development stage, result extraction stage, service development state), each stages has detailed processes(47) and productions(93). SDAD, furthermore, unified previous ISP, DW, SW development methodologies in terms of the data analysis and can easily interchange the productions with them. This paper, lastly, introduces a way to assign responsible persons for each process and provide communication procedures in RACI chart to improves the efficiency of the interaction among professionals from different subjects. SDAD is applied to a Bigdata project in Korea Employment Information Services institution and the result turned out to be acceptable when evaluated by the supervision.

An Analysis of the Relationship between Public Opinion on Social Bigdata and Results after Implementation of Public Policies: A Case Study in 'Welfare' Policy (소셜 빅데이터 기반 공공정책 국민의견 수렴과 정책 시행 이후 결과 관계 분석: '복지' 정책 사례를 중심으로)

  • Kim, Tae-Young;Kim, Yong;Oh, Hyo-Jung
    • Journal of Digital Convergence
    • /
    • v.15 no.3
    • /
    • pp.17-25
    • /
    • 2017
  • Horizon scanning that one of the methods for future prediction is adaptable way of establishing the policy strategy based on big data. This study aims to understand the social problems scientifically utilized horizon scanning technique, and contribute to public policy formulation based on scanning analysis. In this paper, we proposed a public opinion framework for public policy based on social bigdata, and then confirmed the feasibility this framework by analysis of the relationship between public opinion and results after implementation of public policy. Consequently, based on the analysis, we also drew implications of policy formulation about 'free childcare for under 5-years of age' as an object of study. The method that collects public opinion is very important to effective policy establishment and make contribution to constructing national response systems for social development.

Risk Factors Identification and Priority Analysis of Bigdata Project (빅데이터 프로젝트의 위험요인 식별과 우선순위 분석)

  • Kim, Seung-Hee
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.19 no.2
    • /
    • pp.25-40
    • /
    • 2019
  • Many companies are executing big data analysis and utilization projects to legitimize the development of new business areas or conversion of management or technical strategies. In Korea and abroad, however, such projects are failing because they are not completed within specified deadlines, which is not unrelated to the current situation in which the knowledge base for big data project risk management from an engineering perspective is grossly lacking. As such, the current study analyzes the risk factors of big data implementation and utilization projects, in addition to finding risk factors that are highly important. To achieve this end, the study extracts project risk factors via literature review, after which they are grouped using affinity methodology and sifted through expert surveys. The deduced risk factors are structuralize using factor analysis to develop a table that categorizes various types of big data project risk factors. The current study is significant that in it provides a basis for developing basic control indicators related to risk identification, risk assessment, and risk analysis. The findings from the study contribute greatly to the success of big data projects, by providing theoretical basis regarding efficient big data project risk management.