• Title/Summary/Keyword: 실시간 데이터 마이닝

Search Result 145, Processing Time 0.031 seconds

A Sliding Window Technique for Open Data Mining over Data Streams (개방 데이터 마이닝에 효율적인 이동 윈도우 기법)

  • Chang Joong-Hyuk;Lee Won-Suk
    • The KIPS Transactions:PartD
    • /
    • v.12D no.3 s.99
    • /
    • pp.335-344
    • /
    • 2005
  • Recently open data mining methods focusing on a data stream that is a massive unbounded sequence of data elements continuously generated at a rapid rate are proposed actively. Knowledge embedded in a data stream is likely to be changed over time. Therefore, identifying the recent change of the knowledge quickly can provide valuable information for the analysis of the data stream. This paper proposes a sliding window technique for finding recently frequent itemsets, which is applied efficiently in open data mining. In the proposed technique, its memory usage is kept in a small space by delayed-insertion and pruning operations, and its mining result can be found in a short time since the data elements within its target range are not traversed repeatedly. Moreover, the proposed technique focused in the recent data elements, so that it can catch out the recent change of the data stream.

A Study on Continuous Monitoring Reinforcement for Sales Audit Using Process Mining Under Big Data Environment (빅데이터 환경에서 프로세스 마이닝을 이용한 영업감사 상시 모니터링 강화에 대한 연구)

  • Yoo, Young-Seok;Park, Han-Gyu;Back, Seung-Hoon;Hong, Sung-Chan
    • Journal of Internet Computing and Services
    • /
    • v.17 no.6
    • /
    • pp.123-131
    • /
    • 2016
  • Process mining in big data environment utilize a number of data were generated from the business process. It generates lots of knowledge and insights regarding implementation and improvement of the process through the event log of the company's enterprise resource planning (ERP) system. In recent years, various research activities engaged with the audit work of company organizations are trying actively by using the maximum strength of the mining process. However, domestic studies on applicable sales auditing system for the process mining are insufficient under big data environment. Therefore, we propose process-mining methods that can be optimally applied to online and traditional auditing system. In advance, we propose continuous monitoring information system that can early detect and prevent the risk under the big data environment by monitoring risk factors in the organizations of enterprise. The scope of the research of this paper is to design a pre-verification system for risk factor via practical examples in sales auditing. Furthermore, realizations of preventive audit, continuous monitoring for high risk, reduction of fraud, and timely action for violation of rules are enhanced by proposed sales auditing system. According to the simulation results, avoidance of financial risks, reduction of audit period, and improvement of audit quality are represented.

An Alert Data Mining Framework for Intrusion Detection System (침입탐지시스템의 경보데이터 분석을 위한 데이터 마이닝 프레임워크)

  • Shin, Moon-Sun
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.12 no.1
    • /
    • pp.459-466
    • /
    • 2011
  • In this paper, we proposed a data mining framework for the management of alerts in order to improve the performance of the intrusion detection systems. The proposed alert data mining framework performs alert correlation analysis by using mining tasks such as axis-based association rule, axis-based frequent episodes and order-based clustering. It also provides the capability of classify false alarms in order to reduce false alarms. We also analyzed the characteristics of the proposed system through the implementation and evaluation of the proposed system. The proposed alert data mining framework performs not only the alert correlation analysis but also the false alarm classification. The alert data mining framework can find out the unknown patterns of the alerts. It also can be applied to predict attacks in progress and to understand logical steps and strategies behind series of attacks using sequences of clusters and to classify false alerts from intrusion detection system. The final rules that were generated by alert data mining framework can be used to the real time response of the intrusion detection system.

A Study on Social media Opinion Mining based Enterprise Crisis Management (소셜 미디어 오피니언 마이닝에 기반한 기업의 위기관리에 관한 연구)

  • Cha, Seun-Joon;Kang, Jae-Woo;Choi, Jae-Hoon
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2012.06c
    • /
    • pp.142-144
    • /
    • 2012
  • 소셜 미디어가 확산되고 사용자가 증가하면서, 사용자들은 소셜 미디어를 통해 의견을 공유한다. 소셜 미디어는 실시간 정보에 대한 전달이 빠르며 데이터를 수집, 분석할 수 있다. 오피니언 마이닝은 텍스트로부터 사용자의 의견이 포함된 패턴을 추출하여 특정 제품이나 서비스에 대한 의견의 긍정, 부정 표현의 정도를 측정한다. 본 논문에서는 오피니언 마이닝을 기반으로 소셜 미디어 데이터에서 기업의 제품, 서비스와 관련된 사용자의 의견을 분석하여 긍정, 부정인지를 판단한다. 그리고 부정 패턴의 빈도를 통해 기업의 위기 상황을 인지하며, 위기 대응을 위한 4단계의 위기관리 모델을 제시한다. 또한 소셜 미디어에서 기업의 위기관리 사례를 확인하고, 표본조사를 통하여 평가 및 분석을 수행한다. 이 모델을 이용하여 방대한 소셜 미디어 데이터에서 기업의 제품이나 서비스에 대한 부정적 의견을 초기에 감지하고, 체계적으로 대응 할 수 있다.

Efficient Mining for Personalized Medical treatment Diagnosis Service (개인 맞춤형 의료진단 서비스 제공을 위한 효율적인 데이터마이닝 기법)

  • Kaun, Eun-Hee;Lee, Seung-Cheol;Lee, Joo-Chang;Kim, Ung-Mo
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2007.10c
    • /
    • pp.200-204
    • /
    • 2007
  • 최근 유비쿼터스 환경의 발달로 인해 사용자 중심의 유비쿼터스 기술이 활발히 연구되고 있다. 이에 따른 각종 응용 분야가 활발히 연구 중이며, 그 중에서 특히 U-Health 기술이 주목받고 있다. U-Health 기술은 질병의 치료라는 전통적인 관점의 의료 서비스에서 벗어나 건강한 상태의 지속적인 관리와 질병의 예방이라는 적극적이고 확장된 개념으로 발전해가고 있다. 건강상태를 관리하고 진단하기 위해서는 기존의 진단데이터를 효율적으로 관리하고, 그것을 토대로 하여 유용한 정보를 얻어 낼 수 있는 방법이 필요하다. 지금까지는 데이터를 처리하기 위하여 통계적인 수치나 전문가에 의한 전문지식을 토대로 하는 방법을 사용하고 있다. 그러나, 건강상태를 관리하고 진단을 목적으로 하는 시스템에서는 높은 정확성이 보장되어야 한다. 또한 유비쿼터스 환경의 특성상 적은 메모리의 사용과 빠른 마이닝 속도가 수반되어야 한다. 본 논문에서는 튜플기반의 진단데이터들을 마이닝하여 진단패턴을 뽑아내는 의료 진단 마이닝 알고리즘을 제안한다. 본 알고리즘은 진단패턴정보의 정확성을 높일 수 있는 장점을 가지며, 튜플기반의 데이터들을 트리 구조로 구성함으로써 마이닝 속도를 향상시킨다. 더 나아가 트리 구조의 컴팩트한 데이터 구조로 메모리 적재가 용이하다. 이는 센서가 부착된 개별 사용자로부터 실시간으로 들어오는 건강상태와 진단패턴과의 비교, 분석을 가능하게 함으로써 보다 정확하고 빠른 진단결과를 내려줄 수 있는 의사결정시스템의 사용에 적합하다.

  • PDF

A Method for Frequent Itemsets Mining from Data Stream (데이터 스트림 환경에서 효율적인 빈발 항목 집합 탐사 기법)

  • Seo, Bok-Il;Kim, Jae-In;Hwang, Bu-Hyun
    • The KIPS Transactions:PartD
    • /
    • v.19D no.2
    • /
    • pp.139-146
    • /
    • 2012
  • Data Mining is widely used to discover knowledge in many fields. Although there are many methods to discover association rule, most of them are based on frequency-based approaches. Therefore it is not appropriate for stream environment. Because the stream environment has a property that event data are generated continuously. it is expensive to store all data. In this paper, we propose a new method to discover association rules based on stream environment. Our new method is using a variable window for extracting data items. Variable windows have variable size according to the gap of same target event. Our method extracts data using COBJ(Count object) calculation method. FPMDSTN(Frequent pattern Mining over Data Stream using Terminal Node) discovers association rules from the extracted data items. Through experiment, our method is more efficient to apply stream environment than conventional methods.

Building an SNS Crawling System Using Python (Python을 이용한 SNS 크롤링 시스템 구축)

  • Lee, Jong-Hwa
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.23 no.5
    • /
    • pp.61-76
    • /
    • 2018
  • Everything is coming into the world of network where modern people are living. The Internet of Things that attach sensors to objects allows real-time data transfer to and from the network. Mobile devices, essential for modern humans, play an important role in keeping all traces of everyday life in real time. Through the social network services, information acquisition activities and communication activities are left in a huge network in real time. From the business point of view, customer needs analysis begins with SNS data. In this research, we want to build an automatic collection system of SNS contents of web environment in real time using Python. We want to help customers' needs analysis through the typical data collection system of Instagram, Twitter, and YouTube, which has a large number of users worldwide. It is stored in database through the exploitation process and NLP process by using the virtual web browser in the Python web server environment. According to the results of this study, we want to conduct service through the site, the desired data is automatically collected by the search function and the netizen's response can be confirmed in real time. Through time series data analysis. Also, since the search was performed within 5 seconds of the execution result, the advantage of the proposed algorithm is confirmed.

A Design of a TV Advertisement Effectiveness Analysis System Using SNS Big-data (SNS Big-data를 활용한 TV 광고 효과 분석 시스템 설계)

  • Lee, Areum;Bang, Jiseon;Kim, Yoonhee
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.9
    • /
    • pp.579-586
    • /
    • 2015
  • As smart-phone usage increases, the number of Social Networking Service (SNS) users has also exponentially increased. SNS allows people to efficiently exchange their personal opinion, and for this reason, it is possible to collect the reaction of each individual to a given event in real-time. Nevertheless, new methods need to be developed to collect and analyze people's opinion in real-time in order to effectively evaluate the impact of a TV advertisement. Hence, we designed and constructed a system that analyzes the effect of an advertisement in real-time by using data related to the advertisement collected from SNS, specifically, Twitter. In detail, Hadoop is used in the system to enable big-data analysis in parallel, and various analyses can be conducted by conducting separate numerical analyses of the degrees of mentioning, preference and reliability. The analysis can be accurate if the reliability is assessed using opinion mining technology. The proposed system is therefore proven to effectively handle and analyze data responses to divers TV advertisement.

An associative service mining based on dynamic weight (동적 가중치 기반의 연관 서비스 탐사 기법)

  • Hwang, Jeong Hee
    • Journal of Digital Contents Society
    • /
    • v.17 no.5
    • /
    • pp.359-366
    • /
    • 2016
  • In order to provide useful services for user in ubiquitous environment, a technique that can get the helpful information considering user activity and preference is needed and also user's interest actually changes as time passes. Therefore, the discovering method which reflects the concern degree of service information is needed. In this paper, we present the finding method of frequent pattern with dynamic weight on individual item based on service ontology we design. Our method can be applied to provide interested service information for user depending on context.

A design of a Vehicle Analysis System using cloud and data mining (클라우드와 데이터 마이닝을 이용한 차량 분석 시스템 설계)

  • Jeong, Yi-Na;Son, Su-rak;Kim, Kyung-Deuk;Lee, Byung-Kwan
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2019.05a
    • /
    • pp.238-241
    • /
    • 2019
  • In this paper, a "Vehicle Analysis System(VAS) using cloud and data mining" is proposed that store all the sensor data measured in the vehicle in the cloud, analyze the stored data using the classification model, and provide the analyzed data in real time to the driver's display. The VAS consists of two modules. First, Sensor Data Communication Module(SDCM) stores the sensor data measured in the vehicle in a table of the cloud server and transfers the stored data to the analysis module. Second, Sensor Data Analysis Module(SDAM) analyzes the received data using the genetic algorithm and provides analyzed result to the driver in real time. The VAS stores sensor data collected in the vehicle in the cloud server without accumulating it in the vehicle, and stored data is analyzed in the cloud server, so that the sensor data can be quickly and efficiently managed without overloading the vehicle. In addition, the information desired by the driver can be visualized on the display, thereby increasing the stability of the autonomous vehicle.

  • PDF