• Title/Summary/Keyword: Naive Bayes classification

Search Result 125, Processing Time 0.027 seconds

User Information Collection of Weibo Network Public Opinion under Python

  • Changhua Liu;Yanlin Han
    • Journal of Information Processing Systems
    • /
    • v.19 no.3
    • /
    • pp.310-322
    • /
    • 2023
  • Although the network environment is gradually improving, the virtual nature of the network is still the same fact, which has brought a great influence on the supervision of Weibo network public opinion dissemination. In order to reduce this influence, the user information of Weibo network public opinion dissemination is studied by using Python technology. Specifically, the 2019 "Ethiopian air crash" event was taken as the research subject, the relevant data were collected by using Python technology, and the data from March 10, 2019 to June 20, 2019 were constructed by using the implicit Dirichlet distribution topic model and the naive Bayes classifier. The Weibo network public opinion user identity graph model under the "Ethiopian air crash" on June 20 found that the public opinion users of ordinary netizens accounted for the highest proportion and were easily influenced by media public opinion users. This influence is not limited to ordinary netizens. Public opinion users have an influence on other types of public opinion users. That is to say, in the network public opinion space of the "Ethiopian air crash," media public opinion users play an important role in the dissemination of network public opinion information. This research can lay a foundation for the classification and identification of user identity information types under different public opinion life cycles. Future research can start from the supervision of public opinion and the type of user identity to improve the scientific management and control of user information dissemination through Weibo network public opinion.

A New Similarity Measure for Improving Ranking in QA Systems (질의응답시스템 응답순위 개선을 위한 새로운 유사도 계산방법)

  • Kim Myung-Gwan;Park Young-Tack
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.10 no.6
    • /
    • pp.529-536
    • /
    • 2004
  • The main idea of this paper is to combine position information in sentence and query type classification to make the documents ranking to query more accessible. First, the use of conceptual graphs for the representation of document contents In information retrieval is discussed. The method is based on well-known strategies of text comparison, such as Dice Coefficient, with position-based weighted term. Second, we introduce a method for learning query type classification that improves the ability to retrieve answers to questions from Question Answering system. Proposed methods employ naive bayes classification in machine learning fields. And, we used a collection of approximately 30,000 question-answer pairs for training, obtained from Frequently Asked Question(FAQ) files on various subjects. The evaluation on a set of queries from international TREC-9 question answering track shows that the method with machine learning outperforms the underline other systems in TREC-9 (0.29 for mean reciprocal rank and 55.1% for precision).

Variational Bayesian multinomial probit model with Gaussian process classification on mice protein expression level data (가우시안 과정 분류에 대한 변분 베이지안 다항 프로빗 모형: 쥐 단백질 발현 데이터에의 적용)

  • Donghyun Son;Beom Seuk Hwang
    • The Korean Journal of Applied Statistics
    • /
    • v.36 no.2
    • /
    • pp.115-127
    • /
    • 2023
  • Multinomial probit model is a popular model for multiclass classification and choice model. Markov chain Monte Carlo (MCMC) method is widely used for estimating multinomial probit model, but its computational cost is high. However, it is well known that variational Bayesian approximation is more computationally efficient than MCMC, because it uses subsets of samples. In this study, we describe multinomial probit model with Gaussian process classification and how to employ variational Bayesian approximation on the model. This study also compares the results of variational Bayesian multinomial probit model to the results of naive Bayes, K-nearest neighbors and support vector machine for the UCI mice protein expression level data.

Investigating the Performance of Bayesian-based Feature Selection and Classification Approach to Social Media Sentiment Analysis (소셜미디어 감성분석을 위한 베이지안 속성 선택과 분류에 대한 연구)

  • Chang Min Kang;Kyun Sun Eo;Kun Chang Lee
    • Information Systems Review
    • /
    • v.24 no.1
    • /
    • pp.1-19
    • /
    • 2022
  • Social media-based communication has become crucial part of our personal and official lives. Therefore, it is no surprise that social media sentiment analysis has emerged an important way of detecting potential customers' sentiment trends for all kinds of companies. However, social media sentiment analysis suffers from huge number of sentiment features obtained in the process of conducting the sentiment analysis. In this sense, this study proposes a novel method by using Bayesian Network. In this model MBFS (Markov Blanket-based Feature Selection) is used to reduce the number of sentiment features. To show the validity of our proposed model, we utilized online review data from Yelp, a famous social media about restaurant, bars, beauty salons evaluation and recommendation. We used a number of benchmarking feature selection methods like correlation-based feature selection, information gain, and gain ratio. A number of machine learning classifiers were also used for our validation tasks, like TAN, NBN, Sons & Spouses BN (Bayesian Network), Augmented Markov Blanket. Furthermore, we conducted Bayesian Network-based what-if analysis to see how the knowledge map between target node and related explanatory nodes could yield meaningful glimpse into what is going on in sentiments underlying the target dataset.

Ensemble Machine Learning Model Based YouTube Spam Comment Detection (앙상블 머신러닝 모델 기반 유튜브 스팸 댓글 탐지)

  • Jeong, Min Chul;Lee, Jihyeon;Oh, Hayoung
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.24 no.5
    • /
    • pp.576-583
    • /
    • 2020
  • This paper proposes a technique to determine the spam comments on YouTube, which have recently seen tremendous growth. On YouTube, the spammers appeared to promote their channels or videos in popular videos or leave comments unrelated to the video, as it is possible to monetize through advertising. YouTube is running and operating its own spam blocking system, but still has failed to block them properly and efficiently. Therefore, we examined related studies on YouTube spam comment screening and conducted classification experiments with six different machine learning techniques (Decision tree, Logistic regression, Bernoulli Naive Bayes, Random Forest, Support vector machine with linear kernel, Support vector machine with Gaussian kernel) and ensemble model combining these techniques in the comment data from popular music videos - Psy, Katy Perry, LMFAO, Eminem and Shakira.

Impact of Diverse Document-evaluation Measure-based Searching Methods in Big Data Search Accuracy (빅데이터 검색 정확도에 미치는 다양한 측정 방법 기반 검색 기법의 효과)

  • Kim, Ji young;Han, DaHyeon;Kim, Jongkwon
    • Journal of KIISE
    • /
    • v.44 no.5
    • /
    • pp.553-558
    • /
    • 2017
  • With the rapid growth of Big Data, research on extracting meaningful information is being pursued by both academia and industry. Especially, data characteristics derived from analysis, and researcher intention are key factors for search algorithms to obtain accurate output. Therefore, reflecting both data characteristics and researcher intention properly is the final goal of data analysis research. The data analyzed properly can help users to increase loyalty to the service provided by company, and to utilize information more effectively and efficiently. In this paper, we explore various methods of document-evaluation, so that we can improve the accuracy of searching article one of the most frequently searches used in real life. We also analyze the experiment result, and suggest the proper manners to use various methods.

A Study on Injury Severity Prediction for Car-to-Car Traffic Accidents (차대차 교통사고에 대한 상해 심각도 예측 연구)

  • Ko, Changwan;Kim, Hyeonmin;Jeong, Young-Seon;Kim, Jaehee
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.19 no.4
    • /
    • pp.13-29
    • /
    • 2020
  • Automobiles have long been an essential part of daily life, but the social costs of car traffic accidents exceed 9% of the national budget of Korea. Hence, it is necessary to establish prevention and response system for car traffic accidents. In order to present a model that can classify and predict the degree of injury in car traffic accidents, we used big data analysis techniques of K-nearest neighbor, logistic regression analysis, naive bayes classifier, decision tree, and ensemble algorithm. The performances of the models were analyzed by using the data on the nationwide traffic accidents over the past three years. In particular, considering the difference in the number of data among the respective injury severity levels, we used down-sampling methods for the group with a large number of samples to enhance the accuracy of the classification of the models and then verified the statistical significance of the models using ANOVA.

Motion Recognition for Kinect Sensor Data Using Machine Learning Algorithm with PNF Patterns of Upper Extremities

  • Kim, Sangbin;Kim, Giwon;Kim, Junesun
    • The Journal of Korean Physical Therapy
    • /
    • v.27 no.4
    • /
    • pp.214-220
    • /
    • 2015
  • Purpose: The purpose of this study was to investigate the availability of software for rehabilitation with the Kinect sensor by presenting an efficient algorithm based on machine learning when classifying the motion data of the PNF pattern if the subjects were wearing a patient gown. Methods: The motion data of the PNF pattern for upper extremities were collected by Kinect sensor. The data were obtained from 8 normal university students without the limitation of upper extremities. The subjects, wearing a T-shirt, performed the PNF patterns, D1 and D2 flexion, extensions, 30 times; the same protocol was repeated while wearing a patient gown to compare the classification performance of algorithms. For comparison of performance, we chose four algorithms, Naive Bayes Classifier, C4.5, Multilayer Perceptron, and Hidden Markov Model. The motion data for wearing a T-shirt were used for the training set, and 10 fold cross-validation test was performed. The motion data for wearing a gown were used for the test set. Results: The results showed that all of the algorithms performed well with 10 fold cross-validation test. However, when classifying the data with a hospital gown, Hidden Markov model (HMM) was the best algorithm for classifying the motion of PNF. Conclusion: We showed that HMM is the most efficient algorithm that could handle the sequence data related to time. Thus, we suggested that the algorithm which considered the sequence of motion, such as HMM, would be selected when developing software for rehabilitation which required determining the correctness of the motion.

Study of Computer Aided Diagnosis for the Improvement of Survival Rate of Lung Cancer based on Adaboost Learning (폐암 생존율 향상을 위한 아다부스트 학습 기반의 컴퓨터보조 진단방법에 관한 연구)

  • Won, Chulho
    • Journal of rehabilitation welfare engineering & assistive technology
    • /
    • v.10 no.1
    • /
    • pp.87-92
    • /
    • 2016
  • In this paper, we improved classification performance of benign and malignant lung nodules by including the parenchyma features. For small pulmonary nodules (4-10mm) nodules, there are a limited number of CT data voxels within the solid tumor, making them difficult to process through traditional CAD(computer aided diagnosis) tools. Increasing feature extraction to include the surrounding parenchyma will increase the CT voxel set for analysis in these very small pulmonary nodule cases and likely improve diagnostic performance while keeping the CAD tool flexible to scanner model and parameters. In AdaBoost learning using naive Bayes and SVM weak classifier, a number of significant features were selected from 304 features. The results from the COPDGene test yielded an accuracy, sensitivity and specificity of 100%. Therefore proposed method can be used for the computer aided diagnosis effectively.

Development of Multiple Linear Regression Model to Predict Agricultural Reservoir Storage based on Naive Bayes Classification and Weather Forecast Data (나이브 베이즈 분류와 기상예보자료 기반의 농업용 저수지 저수율 전망을 위한 저수율 예측 다중선형 회귀모형 개발)

  • Kim, Jin Uk;Jung, Chung Gil;Lee, Ji Wan;Kim, Seong Joon
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2018.05a
    • /
    • pp.112-112
    • /
    • 2018
  • 최근 이상기후로 인한 국부적인 혹은 광역적인 가뭄이 빈번하게 발생하고 있는 추세이며 발생횟수 뿐 아니라 가뭄 심도 및 지속기간이 과거보다 크게 증가하여 그에 따른 피해가 커질 것으로 예측되고 있다. 특히, 2014~2015년도의 유례없는 가뭄으로 인해 저수지 용수공급이 제한되면서 많은 농가들이 피해를 입었다. 본 연구의 목적은 전국 농업용 저수지를 대상으로 기상청 3개월 예보자료를 활용 할 수 있는 농업용 저수지 저수율 다중선형 회귀 모형을 개발하여 저수율 전망정보를 생산하는 것이다. 본 연구에서는 전국에 적용 가능한 저수율 다중선형 회귀 모형개발을 위해 5개의 기상요소(강수량, 최고기온, 최저기온, 평균기온, 평균풍속)와 관측 저수지 저수율을 활용했다. 기상자료는 2002년부터 2017년까지의 기상청 63개 지상관측소로부터 기상관측자료를 수집하였다. 본 연구에서는 저수율 전망 단계를 세 단계로 나누었다. 첫 번째 단계로 농어촌공사에서 전국 511개 용수구역을 대상으로 군집분석 및 의사결정나무 분석을 통해 제시한 65개 대표저수지를 대상으로 기상자료 및 관측 저수율 자료를 이용하여 다중선형 회귀분석을 실시하였다. 수집한 기상요소와 저수율을 독립변수로 하여 월별 회귀식을 산정한 결과 결정계수($R^2$)는 0.51~0.95로 나타났다. 두 번째 단계로 대표저수지의 회귀분석 결과를 전국의 저수지로 확대하기 위해 나이브 베이즈 분류법을 적용하여 전국 3098개의 저수지를 65의 군집으로 분류하고 각각의 군집에 해당되는 월별 회귀식을 산정하였다. 마지막으로 전국 저수지로 산정된 회귀식과 농업 가뭄 예측을 위해 기상청의 GS5(Global Seasonal Forecasting System 5) 3개월 예보자료를 수집하여 회귀식에 적용해 2017년 전국 저수지의 3개월 저수율 전망정보를 생산하였다. 본 연구의 전국 저수지 군집결과 기반의 저수율 전망기술은 2017년도 관측 저수율과 비교한 결과 유의한 상관성을 나타냈으며 이 결과는 추후 농업용 저수지의 물 공급 및 농업가뭄 전망 자료로서 이용이 가능할 것으로 판단된다.

  • PDF