• 제목/요약/키워드: 가중치 지지도

Search Result 50, Processing Time 0.031 seconds

Frequent Pattern Mining By using a Completeness for BigData (빅데이터에 대한 Completeness를 이용한 빈발 패턴 마이닝)

  • Park, In-Kyu
    • Journal of Korea Game Society
    • /
    • v.18 no.2
    • /
    • pp.121-130
    • /
    • 2018
  • Most of those studies use frequency, the number of times a pattern appears in a transaction database, as the key measure for pattern interestingness. It prerequisites that any interesting pattern should occupy a maximum portion of the transactions it appears. But in our real world scenarios the completeness of any pattern is more likely to become various in transactions. Hence, we should also consider the problem of finding the qualified patterns with the significant values of the weighted support by completeness in order to reduce the loss of information within any pattern in transaction. In these pattern recommendation applications, patterns with higher completeness may lead to higher recall while patterns with higher completeness may lead to higher recall while patterns with higher frequency lead to higher precision. In this paper, we propose a measure of weighted support and completeness and an algorithm WSCFPM(weigted support and completeness frequent pattern mining). Our algorithm handles the invalidation of the monotone or anti-monotone property which does not hold on completeness. Extensive performance analysis show that our algorithm is very efficient and scalable for word pattern mining.

Adaptive Frequent Pattern Algorithm using CAWFP-Tree based on RHadoop Platform (RHadoop 플랫폼기반 CAWFP-Tree를 이용한 적응 빈발 패턴 알고리즘)

  • Park, In-Kyu
    • Journal of Digital Convergence
    • /
    • v.15 no.6
    • /
    • pp.229-236
    • /
    • 2017
  • An efficient frequent pattern algorithm is essential for mining association rules as well as many other mining tasks for convergence with its application spread over a very broad spectrum. Models for mining pattern have been proposed using a FP-tree for storing compressed information about frequent patterns. In this paper, we propose a centroid frequent pattern growth algorithm which we called "CAWFP-Growth" that enhances he FP-Growth algorithm by making the center of weights and frequencies for the itemsets. Because the conventional constraint of maximum weighted support is not necessary to maintain the downward closure property, it is more likely to reduce the search time and the information loss of the frequent patterns. The experimental results show that the proposed algorithm achieves better performance than other algorithms without scarifying the accuracy and increasing the processing time via the centroid of the items. The MapReduce framework model is provided to handle large amounts of data via a pseudo-distributed computing environment. In addition, the modeling of the proposed algorithm is required in the fully distributed mode.

A Study on Negation Handling and Term Weighting Schemes and Their Effects on Mood-based Text Classification (감정 기반 블로그 문서 분류를 위한 부정어 처리 및 단어 가중치 적용 기법의 효과에 대한 연구)

  • Jung, Yu-Chul;Choi, Yoon-Jung;Myaeng, Sung-Hyon
    • Korean Journal of Cognitive Science
    • /
    • v.19 no.4
    • /
    • pp.477-497
    • /
    • 2008
  • Mood classification of blog text is an interesting problem, with a potential for a variety of services involving the Web. This paper introduces an approach to mood classification enhancements through the normalized negation n-grams which contain mood clues and corpus-specific term weighting(CSTW). We've done experiments on blog texts with two different classification methods: Enhanced Mood Flow Analysis(EMFA) and Support Vector Machine based Mood Classification(SVMMC). It proves that the normalized negation n-gram method is quite effective in dealing with negations and gave gradual improvements in mood classification with EMF A. From the selection of CSTW, we noticed that the appropriate weighting scheme is important for supporting adequate levels of mood classification performance because it outperforms the result of TF*IDF and TF.

  • PDF

A MultiDatabase Clustering using Distance of Itemsets (항목집합의 거리를 이용한 다중데이터베이스 클러스터링)

  • Kim, Jin-Hyun;Park, Sung-Lyeon;Youn, Sung-Dae
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2003.05c
    • /
    • pp.1567-1570
    • /
    • 2003
  • 장바구니 데이터들로 구성된 다중데이터베이스를 마이닝 하기 위한 선처리 작업으로는 Ideal&Goodness 기법이 있으며, Ideal&Goodness기법은 유사한 항목이 존재하는 데이터베이스간의 식별이 불가능하다는 단점이 있다. 그러므로 본 논문에서 제안하는 기법은 항목으로만 구성된 집합을 생성하여 데이터베이스간의 거리를 측정하고 항목집합간의 식별능력을 향상시키기 위하여 항목과 지지도를 갖는 항목 데이터 집합을 생성하고 지지도에 대한 확률을 계산한 후, 이를 비교 연산하여 가중치를 계산한다. 본 논문에서는 장바구니 분석을 위한 선처리 단계로써 활용 가능한 클러스터링 기법을 제안하며 성능평가를 통하여 데이터베이스간의 우수한 식별 능력을 보인다.

  • PDF

Extraction of Optimal Moving Patterns of Edge Devices Using Frequencies and Weights (빈발도와 가중치를 적용한 엣지 디바이스의 최적 이동패턴 추출)

  • Lee, YonSik;Jang, MinSeok
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.26 no.5
    • /
    • pp.786-792
    • /
    • 2022
  • In the cloud computing environment, there has been a lot of research into the Fog/Edge Computing (FEC) paradigm for securing user proximity of application services and computation offloading to alleviate service delay difficulties. The method of predicting dynamic location change patterns of edge devices (moving objects) requesting application services is critical in this FEC environment for efficient computing resource distribution and deployment. This paper proposes an optimal moving pattern extraction algorithm in which variable weights (distance, time, congestion) are applied to selected paths in addition to a support factor threshold for frequency patterns (moving objects) of edge devices. The proposed algorithm is compared to the OPE_freq [8] algorithm, which just applies frequency, as well as the A* and Dijkstra algorithms, and it can be shown that the execution time and number of nodes accessed are reduced, and a more accurate path is extracted through experiments.

Weighted association rules considering item RFM scores (항목 알에프엠 점수를 고려한 가중 연관성 규칙)

  • Park, Hee-Chang
    • Journal of the Korean Data and Information Science Society
    • /
    • v.21 no.6
    • /
    • pp.1147-1154
    • /
    • 2010
  • One of the important goals in data mining is to discover and decide the relationships between different variables. Association rules are required for this technique and it find meaningful rules by quantifying the relationship between two items based on association measures such as support, confidence, and lift. In this paper, we presented the evaluation criteria of weighted association rule considering item RFM scores as importance of items. Original RFM technique has been used most widely applied method using customer information to find the most profitable customers. And then we compared general association rule technique with weighted association rule technique through the simulation data.

Spam Filter by Using X2 Statistics and Support Vector Machines (카이제곱 통계량과 지지벡터기계를 이용한 스팸메일 필터)

  • Lee, Song-Wook
    • The KIPS Transactions:PartB
    • /
    • v.17B no.3
    • /
    • pp.249-254
    • /
    • 2010
  • We propose an automatic spam filter for e-mail data using Support Vector Machines(SVM). We use a lexical form of a word and its part of speech(POS) tags as features and select features by chi square statistics. We represent each feature by TF(text frequency), TF-IDF, and binary weight for experiments. After training SVM with the selected features, SVM classifies each e-mail as spam or not. In experiment, the selected features improve the performance of our system and we acquired overall 98.9% of accuracy with TREC05-p1 spam corpus.

A Document Sentiment Classification System Based on the Feature Weighting Method Improved by Measuring Sentence Sentiment Intensity (문장 감정 강도를 반영한 개선된 자질 가중치 기법 기반의 문서 감정 분류 시스템)

  • Hwang, Jae-Won;Ko, Young-Joong
    • Journal of KIISE:Software and Applications
    • /
    • v.36 no.6
    • /
    • pp.491-497
    • /
    • 2009
  • This paper proposes a new feature weighting method for document sentiment classification. The proposed method considers the difference of sentiment intensities among sentences in a document. Sentiment features consist of sentiment vocabulary words and the sentiment intensity scores of them are estimated by the chi-square statistics. Sentiment intensity of each sentence can be measured by using the obtained chi-square statistics value of each sentiment feature. The calculated intensity values of each sentence are finally applied to the TF-IDF weighting method for whole features in the document. In this paper, we evaluate the proposed method using support vector machine. Our experimental results show that the proposed method performs about 2.0% better than the baseline which doesn't consider the sentiment intensity of a sentence.

A Semantic Orientation Prediction Method of Sentiment Features Based on the General and Domain-Dependent Characteristics (일반적, 영역 의존적 특성을 반영한 감정 자질의 의미지향성 추정 방법)

  • Hwang, Jaewon;Ko, Youngjoong
    • Annual Conference on Human and Language Technology
    • /
    • 2009.10a
    • /
    • pp.155-159
    • /
    • 2009
  • 본 논문은 한국어 문서 감정분류를 위한 중요한 어휘 자원인 감정자질(Sentiment Feature)의 의미지향성(Semantic Orientation) 추정을 위해 일반적인 특성과 영역(Domain) 의존적인 특성을 반영하여 한국어 문서 감정분류(Sentiment Classification)의 성능 향상을 얻을 수 있는 기법을 제안한다. 감정자질의 의미지 향성은 검색 엔진을 통해 추출한 각 감정 자질의 스니핏(Snippet)과 실험 말뭉치를 이용하여 추정할 수 있다. 검색 엔진을 통해 추출된 스니핏은 감정자질의 일반적인 특성을 반영하며, 실험 말뭉치는 분류하고자 하는 영역 의존적인 특성을 반영한다. 이렇게 얻어진 감정자질의 의미지향성 수치는 각 문장의 감정강도를 추정하기 위해 이용되며, 문장의 감정 강도의 값을 TF-IDF 가중치 기법에 접목하여 감정자질의 가중치를 책정한다. 최종적으로 학습 과정에서 긍정 문서에서는 긍정 감정자질, 부정 문서에서는 부정 감정자질을 대상으로 추가 가중치를 부여하여 학습하였다. 본 논문에서는 문서 분류에 뛰어난 성능을 보여주는 지지 벡터 기계(Support Vector Machine)를 사용하여 제안한 방법의 성능을 평가한다. 평가 결과, 일반적인 정보 검색에서 사용하는 내용어(Content Word) 기반의 자질을 사용한 경우보다 3.1%의 성능향상을 보였다.

  • PDF

Association Rule Mining Scheme of Large-Scale Database for Socially Aware Computing (Socially aware computing을 위한 대규모 데이터베이스의 연관 규칙 감축 기법)

  • Jeong, Hwi-Woon;Park, Geon-Yong;Park, Jong-Chang;Youn, Hee-Yong
    • Proceedings of the Korean Society of Computer Information Conference
    • /
    • 2013.01a
    • /
    • pp.291-294
    • /
    • 2013
  • 연관 규칙 감축 기법은 대규모 데이터를 사용하는 Socially aware computing분야에서 매우 중요한 이슈이다. 본 논문에서는 수집된 각종 데이터들을 각 속성 기준에 따라 이진 변환한 후 가중치를 부여하고 논리식 감축 방법을 이용하여 신뢰성을 보장하는 규칙을 도출하는 새로운 데이터 감축 기법을 제안한다. 이는 컴퓨터 시뮬레이션 결과 기존의 방식들에 비해 지지도, 신뢰도, 규칙 감소율, 연관 규칙 추출 시간에 좋은 성능을 보였으며 이는 빠른 시간 내에 신뢰성 높은 대규모 데이터 처리가 필요한 Socially aware computing분야에 적합하다고 판단한다.

  • PDF