• Title/Summary/Keyword: 빈발 패턴

Search Result 128, Processing Time 0.038 seconds

Frequently Occurred Information Extraction from a Collection of Labeled Trees (라벨 트리 데이터의 빈번하게 발생하는 정보 추출)

  • Paik, Ju-Ryon;Nam, Jung-Hyun;Ahn, Sung-Joon;Kim, Ung-Mo
    • Journal of Internet Computing and Services
    • /
    • v.10 no.5
    • /
    • pp.65-78
    • /
    • 2009
  • The most commonly adopted approach to find valuable information from tree data is to extract frequently occurring subtree patterns from them. Because mining frequent tree patterns has a wide range of applications such as xml mining, web usage mining, bioinformatics, and network multicast routing, many algorithms have been recently proposed to find the patterns. However, existing tree mining algorithms suffer from several serious pitfalls in finding frequent tree patterns from massive tree datasets. Some of the major problems are due to (1) modeling data as hierarchical tree structure, (2) the computationally high cost of the candidate maintenance, (3) the repetitious input dataset scans, and (4) the high memory dependency. These problems stem from that most of these algorithms are based on the well-known apriori algorithm and have used anti-monotone property for candidate generation and frequency counting in their algorithms. To solve the problems, we base a pattern-growth approach rather than the apriori approach, and choose to extract maximal frequent subtree patterns instead of frequent subtree patterns. The proposed method not only gets rid of the process for infrequent subtrees pruning, but also totally eliminates the problem of generating candidate subtrees. Hence, it significantly improves the whole mining process.

  • PDF

Advanced Improvement for Frequent Pattern Mining using Bit-Clustering (비트 클러스터링을 이용한 빈발 패턴 탐사의 성능 개선 방안)

  • Kim, Eui-Chan;Kim, Kye-Hyun;Lee, Chul-Yong;Park, Eun-Ji
    • Journal of Korea Spatial Information System Society
    • /
    • v.9 no.1
    • /
    • pp.105-115
    • /
    • 2007
  • Data mining extracts interesting knowledge from a large database. Among numerous data mining techniques, research work is primarily concentrated on clustering and association rules. The clustering technique of the active research topics mainly deals with analyzing spatial and attribute data. And, the technique of association rules deals with identifying frequent patterns. There was an advanced apriori algorithm using an existing bit-clustering algorithm. In an effort to identify an alternative algorithm to improve apriori, we investigated FP-Growth and discussed the possibility of adopting bit-clustering as the alternative method to solve the problems with FP-Growth. FP-Growth using bit-clustering demonstrated better performance than the existing method. We used chess data in our experiments. Chess data were used in the pattern mining evaluation. We made a creation of FP-Tree with different minimum support values. In the case of high minimum support values, similar results that the existing techniques demonstrated were obtained. In other cases, however, the performance of the technique proposed in this paper showed better results in comparison with the existing technique. As a result, the technique proposed in this paper was considered to lead to higher performance. In addition, the method to apply bit-clustering to GML data was proposed.

  • PDF

An Alert Data Mining Framework for Intrusion Detection System (침입탐지시스템의 경보데이터 분석을 위한 데이터 마이닝 프레임워크)

  • Shin, Moon-Sun
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.12 no.1
    • /
    • pp.459-466
    • /
    • 2011
  • In this paper, we proposed a data mining framework for the management of alerts in order to improve the performance of the intrusion detection systems. The proposed alert data mining framework performs alert correlation analysis by using mining tasks such as axis-based association rule, axis-based frequent episodes and order-based clustering. It also provides the capability of classify false alarms in order to reduce false alarms. We also analyzed the characteristics of the proposed system through the implementation and evaluation of the proposed system. The proposed alert data mining framework performs not only the alert correlation analysis but also the false alarm classification. The alert data mining framework can find out the unknown patterns of the alerts. It also can be applied to predict attacks in progress and to understand logical steps and strategies behind series of attacks using sequences of clusters and to classify false alerts from intrusion detection system. The final rules that were generated by alert data mining framework can be used to the real time response of the intrusion detection system.

Implementation of Analyzer of the Alert Data using Data Mining (데이타마이닝 기법을 이용한 경보데이타 분석기 구현)

  • 신문선;김은희;문호성;류근호;김기영
    • Journal of KIISE:Databases
    • /
    • v.31 no.1
    • /
    • pp.1-12
    • /
    • 2004
  • As network systems are developed rapidly and network architectures are more complex than before, it needs to use PBNM(Policy-Based Network Management) in network system. Generally, architecture of the PBNM consists of two hierarchical layers: management layer and enforcement layer. A security policy server in the management layer should be able to generate new policy, delete, update the existing policy and decide the policy when security policy is requested. And the security policy server should be able to analyze and manage the alert messages received from Policy enforcement system in the enforcement layer for the available information. In this paper, we propose an alert analyzer using data mining. First, in the framework of the policy-based network security management, we design and implement an alert analyzes that analyzes alert data stored in DBMS. The alert analyzer is a helpful system to manage the fault users or hosts. Second, we implement a data mining system for analyzing alert data. The implemented mining system can support alert analyzer and the high level analyzer efficiently for the security policy management. Finally, the proposed system is evaluated with performance parameter, and is able to find out new alert sequences and similar alert patterns.

Web Structure Mining by Extracting Hyperlinks from Web Documents and Access Logs (웹 문서와 접근로그의 하이퍼링크 추출을 통한 웹 구조 마이닝)

  • Lee, Seong-Dae;Park, Hyu-Chan
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.11 no.11
    • /
    • pp.2059-2071
    • /
    • 2007
  • If the correct structure of Web site is known, the information provider can discover users# behavior patterns and characteristics for better services, and users can find useful information easily and exactly. There may be some difficulties, however, to extract the exact structure of Web site because documents one the Web tend to be changed frequently. This paper proposes new method for extracting such Web structure automatically. The method consists of two phases. The first phase extracts the hyperlinks among Web documents, and then constructs a directed graph to represent the structure of Web site. It has limitations, however, to discover the hyperlinks in Flash and Java Applet. The second phase is to find such hidden hyperlinks by using Web access log. It fist extracts the click streams from the access log, and then extract the hidden hyperlinks by comparing with the directed graph. Several experiments have been conducted to evaluate the proposed method.

Development of locally customized river level prediction model based on AI for ASEAN countries (ASEAN국가 현지맞춤형 인공지능 하천수위예측 모형 개발)

  • Sooyoung Kim;Jaewon Jung;Seungho Lee;Kwang-Seok Yoon
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2023.05a
    • /
    • pp.333-333
    • /
    • 2023
  • 기후변화로 인해 전지구적인 이상기후현상이 빈번하게 발생하고 있으며 지구온도와 해수면상승과 더불어 강우패턴의 변화가 세계적인 문제로 대두되고 있다. 특히 아세안 국가의 경우 태풍 및 집중호우에 대한 침수피해의 빈발로 2,000만명 이상이 피해를 입은 것으로 나타났다. 이러한 윈인으로는 자연재해에 의한 인명 및 재산피해관련 대응기술의 개발 및 대응조직의 전문성이 미흡하다는 것이 가장 큰 원인으로 제시되고 있다. 이에 많은 국가 및 기관에서 재난 대응 기술을 ODA사업을 통해 지원하고 있다. 우리나라에서도 지속적인 ODA사업으로 재난 대응 기술, 그 중에서도 홍수대응 기술을 적극적으로 보급하고 있다. 본 연구에서는 ASEAN국가 현지 맞춤형 인공지능 하천수위예측 모형을 개발하여 ASEAN국가의 홍수대응 능력을 향상시키고자 하였다. 연구대상으로는 관측데이터의 수집이 용이하고 양질의 관측자료를 장기간 확보할 수 있는 필리핀의 Montalban 관측소를 대상으로 하였다. Montalban 수위관측소는 마닐라를 관통하는 마리키나 강의 상류에 위치하고 있다. 주변에는 상류쪽에 Mt. Oro 강우관측소가 있으며 해당 관측소의 강우자료와 Montalban 관측소의 수위자료를 입력자료로 활용하여 최대 3시간까지 수위를 예측하였다. 예측수위에 대한 적합도 지표로 NSE(Nash-Sutcliffe model efficiency coefficient)를 사용하였으며 2시간 예측까지는 0.8이상의 유의미한 결과를 나타내 홍수예보에 활용할 수 있을 것으로 판단되나, 3시간 예측결과는 홍수예보에 활용하기 어려운 것으로 판단하였다. 이는 Mt. Oro관측소에 내린 강우가 Montalban 관측소에 도달하기까지 소요되는 시간이 3시간 이내이기 때문으로 판단된다. 관측소의 수위자료와 상류에 위치한 강우관측소의 장기간 고품질의 관측자료가 존재한다면 높은 정확도의 예측결과를 도출 할 수 있을 것으로 판단된다.

  • PDF

Implementation of Role-based Command Hierarchy Model for Actor Cooperation (ROCH: 워게임 모의개체 간 역할기반 협력 구현 방안 연구)

  • Kim, Jungyoon;Kim, Hee-Soo;Lee, Sangjin
    • Journal of the Korea Society for Simulation
    • /
    • v.24 no.4
    • /
    • pp.107-118
    • /
    • 2015
  • Many approaches to agent collaboration have been introduced in military war-games, and those approaches address methods for simulation entity (actor) collaboration within a team to achieve given goals. To meet fast-changing battlefield situations, an actor must be loosely coupled with their tasks and be able to take over the role of other actors if necessary to reflect role handovers occurring in real combat. Achieving these requirements allows the transfer of tasks assigned one actor to another actor in circumstances when that actor cannot execute its assigned role, such as when destroyed in action. Tight coupling between an actor and its tasks can prevent role handover in fast-changing situations. Unfortunately, existing approaches and war-game strictly assign tasks to actors during design, therefore they prevent the loose coupling. To overcome these shortcomings, our Role-based Command Hierarchy (ROCH) model dynamically assigns roles to actors based on their situation at runtime. In the model, "Role" separates actors from their tasks. In this paper, we implement the ROCH model as a component that uses a publish-subscribe pattern to handle the link between an actor and the roles of its subordinates (other actors).

Development of Intelligent Job Classification System based on Job Posting on Job Sites (구인구직사이트의 구인정보 기반 지능형 직무분류체계의 구축)

  • Lee, Jung Seung
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.4
    • /
    • pp.123-139
    • /
    • 2019
  • The job classification system of major job sites differs from site to site and is different from the job classification system of the 'SQF(Sectoral Qualifications Framework)' proposed by the SW field. Therefore, a new job classification system is needed for SW companies, SW job seekers, and job sites to understand. The purpose of this study is to establish a standard job classification system that reflects market demand by analyzing SQF based on job offer information of major job sites and the NCS(National Competency Standards). For this purpose, the association analysis between occupations of major job sites is conducted and the association rule between SQF and occupation is conducted to derive the association rule between occupations. Using this association rule, we proposed an intelligent job classification system based on data mapping the job classification system of major job sites and SQF and job classification system. First, major job sites are selected to obtain information on the job classification system of the SW market. Then We identify ways to collect job information from each site and collect data through open API. Focusing on the relationship between the data, filtering only the job information posted on each job site at the same time, other job information is deleted. Next, we will map the job classification system between job sites using the association rules derived from the association analysis. We will complete the mapping between these market segments, discuss with the experts, further map the SQF, and finally propose a new job classification system. As a result, more than 30,000 job listings were collected in XML format using open API in 'WORKNET,' 'JOBKOREA,' and 'saramin', which are the main job sites in Korea. After filtering out about 900 job postings simultaneously posted on multiple job sites, 800 association rules were derived by applying the Apriori algorithm, which is a frequent pattern mining. Based on 800 related rules, the job classification system of WORKNET, JOBKOREA, and saramin and the SQF job classification system were mapped and classified into 1st and 4th stages. In the new job taxonomy, the first primary class, IT consulting, computer system, network, and security related job system, consisted of three secondary classifications, five tertiary classifications, and five fourth classifications. The second primary classification, the database and the job system related to system operation, consisted of three secondary classifications, three tertiary classifications, and four fourth classifications. The third primary category, Web Planning, Web Programming, Web Design, and Game, was composed of four secondary classifications, nine tertiary classifications, and two fourth classifications. The last primary classification, job systems related to ICT management, computer and communication engineering technology, consisted of three secondary classifications and six tertiary classifications. In particular, the new job classification system has a relatively flexible stage of classification, unlike other existing classification systems. WORKNET divides jobs into third categories, JOBKOREA divides jobs into second categories, and the subdivided jobs into keywords. saramin divided the job into the second classification, and the subdivided the job into keyword form. The newly proposed standard job classification system accepts some keyword-based jobs, and treats some product names as jobs. In the classification system, not only are jobs suspended in the second classification, but there are also jobs that are subdivided into the fourth classification. This reflected the idea that not all jobs could be broken down into the same steps. We also proposed a combination of rules and experts' opinions from market data collected and conducted associative analysis. Therefore, the newly proposed job classification system can be regarded as a data-based intelligent job classification system that reflects the market demand, unlike the existing job classification system. This study is meaningful in that it suggests a new job classification system that reflects market demand by attempting mapping between occupations based on data through the association analysis between occupations rather than intuition of some experts. However, this study has a limitation in that it cannot fully reflect the market demand that changes over time because the data collection point is temporary. As market demands change over time, including seasonal factors and major corporate public recruitment timings, continuous data monitoring and repeated experiments are needed to achieve more accurate matching. The results of this study can be used to suggest the direction of improvement of SQF in the SW industry in the future, and it is expected to be transferred to other industries with the experience of success in the SW industry.