• Title/Summary/Keyword: TF-IDF analysis

Search Result 197, Processing Time 0.023 seconds

Analysis of Major COVID-19 Issues Using Unstructured Big Data (비정형 빅데이터를 이용한 COVID-19 주요 이슈 분석)

  • Kim, Jinsol;Shin, Donghoon;Kim, Heewoong
    • Knowledge Management Research
    • /
    • v.22 no.2
    • /
    • pp.145-165
    • /
    • 2021
  • As of late December 2019, the spread of COVID-19 pandemic began which put the entire world in panic. In order to overcome the crisis and minimize any subsequent damage, the government as well as its affiliated institutions must maximize effects of pre-existing policy support and introduce a holistic response plan that can reflect this changing situation- which is why it is crucial to analyze social topics and people's interests. This study investigates people's major thoughts, attitudes and topics surrounding COVID-19 pandemic through the use of social media and big data. In order to collect public opinion, this study segmented time period according to government countermeasures. All data were collected through NAVER blog from 31 December 2019 to 12 December 2020. This research applied TF-IDF keyword extraction and LDA topic modeling as text-mining techniques. As a result, eight major issues related to COVID-19 have been derived, and based on these keywords, this research presented policy strategies. The significance of this study is that it provides a baseline data for Korean government authorities in providing appropriate countermeasures that can satisfy needs of people in the midst of COVID-19 pandemic.

A New Scheme Exploiting the Related Keyword and Big Data Analysis for Predicting Promise Technology in the Field of Satellite·Terrestrial Information Convergence Disaster Response (위성·지상정보 융합 재난 대응 기술 분야 유망기술 도출을 위한 연관 키워드 및 빅데이터 분석 기법)

  • Lee, Hangwon;Kim, Youngok
    • Journal of the Society of Disaster Information
    • /
    • v.18 no.2
    • /
    • pp.418-431
    • /
    • 2022
  • Purpose: We propose a new scheme for predicting promise technology and it improves the conventional scheme that misses important lists of patent because of insufficient search formula, and cannot reflect new trend of technology due to the unreleased period of patents. Method: In this paper, we propose a new search formula exploiting TF and TF-IDF with R programming as well as related keywords, and LDA topic modeling scheme is used for analyzing recently published papers in Satellite·Terrestrial Information Convergence Disaster Response. Result: By comparing both schemes with commercial DB, the proposed scheme can find more important patents, and can reflect new trend of technology, compared to the conventional scheme. Conclusion: The proposed scheme can be used to predict promise technologies in the field of Satellite·Terrestrial Information Convergence Disaster Response.

A Study on the Current Situation and Trend Analysis of The Elderly Healthcare Applications Using Big Data Analysis (텍스트마이닝을 활용한 노인 헬스케어 앱 사용 추이 및 동향 분석)

  • Byun, Hyun;Jeon, Sang-Wan;YI, Eun-Surk
    • Journal of the Korea Convergence Society
    • /
    • v.13 no.5
    • /
    • pp.313-325
    • /
    • 2022
  • The purpose of this study is to examine the changes in the elderly healthcare app market through text mining analysis and to present basic data for activating elderly healthcare apps. Data collection was conducted on Naver, Daum, blog web, and cafe. As for the research method, text mining, TF-IDF(Term frequency-inverse document frequency), emotional analysis, and semantic network analysis were conducted using Textom and Ucinet6, which are big data analysis programs. As a result of this study, a total of six categories were finally derived: resolving the healthcare app information gap, convergence healthcare technology, diffusion media, elderly healthcare app industry, social background, and content. In conclusion, in order for elderly healthcare apps to be accepted and utilized by the elderly, they must have a good diffusion infrastructure, and the effectiveness of healthcare apps must be maximized through the active introduction of convergence technology and content development that can be easily used by the elderly.

Music information and musical propensity analysis, and music recommendation system using collaborative filtering (음악정보와 음악적 성향 분석 및 협업 필터링을 이용한 음악추천시스템)

  • Gong, Minseo;Hong, Jinju;Choi, Jaehyun
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2015.05a
    • /
    • pp.533-536
    • /
    • 2015
  • Mobile music market is growing. However, services what are applied recently are inaccurate to recommend music that a user is worth to prefer. So, this paper suggests music recommend system. This system recommend music that users prefer analyzing music information and user's musical propensity and using collaborative filtering. This system classify genre and extract factors what can be get using STFT's ZCR, Spectral roll-off, Spectral flux. So similar musics are clustered by these factors. And then, after divide mood of music's lyric, it finally recommend music automatically using collaborative filtering.

  • PDF

An Effective Metric for Measuring the Degree of Web Page Changes (효과적인 웹 문서 변경도 측정 방법)

  • Kwon, Shin-Young;Kim, Sung-Jin;Lee, Sang-Ho
    • Journal of KIISE:Databases
    • /
    • v.34 no.5
    • /
    • pp.437-447
    • /
    • 2007
  • A variety of similarity metrics have been used to measure the degree of web page changes. In this paper, we first define criteria for web page changes to evaluate the effectiveness of the similarity metrics in terms of six important types of web page changes. Second, we propose a new similarity metric appropriate for measuring the degree of web page changes. Using real web pages and synthesized pages, we analyze the five existing metrics (i.e., the byte-wise comparison, the TF IDF cosine distance, the word distance, the edit distance, and the shingling) and ours under the proposed criteria. The analysis result shows that our metric represents the changes more effectively than other metrics. We expect that our study can help users select an appropriate metric for particular web applications.

Predicting numeric ratings for Google apps using text features and ensemble learning

  • Umer, Muhammad;Ashraf, Imran;Mehmood, Arif;Ullah, Saleem;Choi, Gyu Sang
    • ETRI Journal
    • /
    • v.43 no.1
    • /
    • pp.95-108
    • /
    • 2021
  • Application (app) ratings are feedback provided voluntarily by users and serve as important evaluation criteria for apps. However, these ratings can often be biased owing to insufficient or missing votes. Additionally, significant differences have been observed between numeric ratings and user reviews. This study aims to predict the numeric ratings of Google apps using machine learning classifiers. It exploits numeric app ratings provided by users as training data and returns authentic mobile app ratings by analyzing user reviews. An ensemble learning model is proposed for this purpose that considers term frequency/inverse document frequency (TF/IDF) features. Three TF/IDF features, including unigrams, bigrams, and trigrams, were used. The dataset was scraped from the Google Play store, extracting data from 14 different app categories. Biased and unbiased user ratings were discriminated using TextBlob analysis to formulate the ground truth, from which the classifier prediction accuracy was then evaluated. The results demonstrate the high potential for machine learning-based classifiers to predict authentic numeric ratings based on actual user reviews.

Social perception of the Arduino lecture as seen in big data (빅데이터 분석을 통한 아두이노 강의에 대한 사회적 인식)

  • Lee, Eunsang
    • Journal of The Korean Association of Information Education
    • /
    • v.25 no.6
    • /
    • pp.935-945
    • /
    • 2021
  • The purpose of this study is to analyze the social perception of Arduino lecture using big data analysis method. For this purpose, data from January 2012 to May 2021 were collected using the Textom website as a keyword searched for 'arduino + lecture' in blogs, cafes, and news channels of NAVER website. The collected data was refined using the Textom website, and text mining analysis and semantic network analysis were performed by opening the Textom website, Ucinet 6, and Netdraw programs. As a result of text mining analysis such as frequency analysis, TF-IDF analysis, and degree centrality it was confirmed that 'education' and 'coding' were the top keywords. As a result of CONCOR analysis for semantic network analysis, four clusters can be identified: 'Arduino-related education', 'Physical computing-related lecture', 'Arduino special lecture', and 'GUI programming'. Through this study, it was possible to confirm various meaningful social perceptions of the general public in relation to Arduino lecture on the Internet. The results of this study will be used as data that provides meaningful implications for instructors preparing for Arduino lectures, researchers studying the subject, and policy makers who establish software education or coding education and related policies.

A Study on the Analysis of Related Information through the Establishment of the National Core Technology Network: Focused on Display Technology (국가핵심기술 관계망 구축을 통한 연관정보 분석연구: 디스플레이 기술을 중심으로)

  • Pak, Se Hee;Yoon, Won Seok;Chang, Hang Bae
    • The Journal of Society for e-Business Studies
    • /
    • v.26 no.2
    • /
    • pp.123-141
    • /
    • 2021
  • As the dependence of technology on the economic structure increases, the importance of National Core Technology is increasing. However, due to the nature of the technology itself, it is difficult to determine the scope of the technology to be protected because the scope of the relation is abstract and information disclosure is limited due to the nature of the National Core Technology. To solve this problem, we propose the most appropriate literature type and method of analysis to distinguish important technologies related to National Core Technology. We conducted a pilot test to apply TF-IDF, and LDA topic modeling, two techniques of text mining analysis for big data analysis, to four types of literature (news, papers, reports, patents) collected with National Core Technology keywords in the field of Display industry. As a result, applying LDA theme modeling to patent data are highly relevant to National Core Technology. Important technologies related to the front and rear industries of displays, including OLEDs and microLEDs, were identified, and the results were visualized as networks to clarify the scope of important technologies associated with National Core Technology. Throughout this study, we have clarified the ambiguity of the scope of association of technologies and overcome the limited information disclosure characteristics of national core technologies.

Analysis of Meta Fashion Meaning Structure using Big Data: Focusing on the keywords 'Metaverse' + 'Fashion design' (빅데이터를 활용한 메타패션 의미구조 분석에 관한 연구: '메타버스' + '패션디자인' 키워드를 중심으로)

  • Ji-Yeon Kim;Shin-Young Lee
    • Fashion & Textile Research Journal
    • /
    • v.25 no.5
    • /
    • pp.549-559
    • /
    • 2023
  • Along with the transition to the fourth industrial revolution, the possibility of metaverse-based innovation in the fashion field has been confirmed, and various applications are being sought. Therefore, this study performs meaning structure analysis and discusses the prospects of meta fashion using big data. From 2020 to 2022, data including the keyword "metaverse + fashion design" were collected from portal sites (Naver, Daum, and Google), and the results of keyword frequency, N-gram, and TF-IDF analyses were derived using text mining. Furthermore, network visualization and CONCOR analysis were performed using Ucinet 6 to understand the interconnected structure between keywords and their essential meanings. The results were as follows: The main keywords appeared in the following order: fashion, metaverse, design, 3D, platform, apparel, and virtual. In the N-gram analysis, the density between fashion and metaverse words was high, and in the TF-IDF analysis results, the importance of content- and technology-related words such as 3D, apparel, platform, NFT, education, AI, avatar, MCM, and meta-fashion was confirmed. Through network visualization and CONCOR analysis using Ucinet 6, three cluster results were derived from the top emerging words: "metaverse fashion design and industry," "metaverse fashion design and education," and "metaverse fashion design platform." CONCOR analysis was also used to derive differentiated analysis results for middle and lower words. The results of this study provide useful information to strengthen competitiveness in the field of metaverse fashion design.

A Study on Monitoring Method of Citizen Opinion based on Big Data : Focused on Gyeonggi Lacal Currency (Gyeonggi Money) (빅데이터 기반 시민의견 모니터링 방안 연구 : "경기지역화폐"를 중심으로)

  • Ahn, Soon-Jae;Lee, Sae-Mi;Ryu, Seung-Ei
    • Journal of Digital Convergence
    • /
    • v.18 no.7
    • /
    • pp.93-99
    • /
    • 2020
  • Text mining is one of the big data analysis methods that extracts meaningful information from atypical large-scale text data. In this study, text mining was used to monitor citizens' opinions on the policies and systems being implemented. We collected 5,108 newspaper articles and 748 online cafe posts related to 'Gyeonggi Lacal Currency' and performed frequency analysis, TF-IDF analysis, association analysis, and word tree visualization analysis. As a result, many articles related to the purpose of introducing local currency, the benefits provided, and the method of use. However, the contents related to the actual use of local currency were written in the online cafe posts. In order to revitalize local currency, the news was involved in the promotion of local currency as an informant. Online cafe posts consisted of the opinions of citizens who are local currency users. SNS and text mining are expected to effectively activate various policies as well as local currency.