• Title/Summary/Keyword: Topic Data

Search Result 1,572, Processing Time 0.025 seconds

Futures Price Prediction based on News Articles using LDA and LSTM (LDA와 LSTM를 응용한 뉴스 기사 기반 선물가격 예측)

  • Jin-Hyeon Joo;Keun-Deok Park
    • Journal of Industrial Convergence
    • /
    • v.21 no.1
    • /
    • pp.167-173
    • /
    • 2023
  • As research has been published to predict future data using regression analysis or artificial intelligence as a method of analyzing economic indicators. In this study, we designed a system that predicts prospective futures prices using artificial intelligence that utilizes topic probability data obtained from past news articles using topic modeling. Topic probability distribution data for each news article were obtained using the Latent Dirichlet Allocation (LDA) method that can extract the topic of a document from past news articles via unsupervised learning. Further, the topic probability distribution data were used as the input for a Long Short-Term Memory (LSTM) network, a derivative of Recurrent Neural Networks (RNN) in artificial intelligence, in order to predict prospective futures prices. The method proposed in this study was able to predict the trend of futures prices. Later, this method will also be able to predict the trend of prices for derivative products like options. However, because statistical errors occurred for certain data; further research is required to improve accuracy.

A Text Mining Study on Endangered Wildlife Complaints - Discovery of Key Issues through LDA Topic Modeling and Network Analysis - (멸종위기 야생생물 민원 텍스트 마이닝 연구 - LDA 토픽 모델링과 네트워크 분석을 통한 주요 이슈 발굴 -)

  • Kim, Na-Yeong;Nam, Hee-Jung;Park, Yong-Su
    • Journal of the Korean Society of Environmental Restoration Technology
    • /
    • v.26 no.6
    • /
    • pp.205-220
    • /
    • 2023
  • This study aimed to analyze the needs and interests of the public on endangered wildlife using complaint big data. We collected 1,203 complaints and their corresponding text data on endangered wildlife, pre-processed them, and constructed a document-term matrix for 1,739 text data. We performed LDA (Latent Dirichlet Allocation) topic modeling and network analysis. The results revealed that the complaints on endangered wildlife peaked in June-August, and the interest shifted from insects to various endangered wildlife in the living area, such as mammals, birds, and amphibians. In addition, the complaints on endangered wildlife could be categorized into 8 topics and 5 clusters, such as discovery report, habitat protection and response request, information inquiry, investigation and action request, and consultation request. The co-occurrence network analysis for each topic showed that the keywords reflecting the call center reporting procedure, such as photo, send, and take, had high centrality in common, and other keywords such as dung beetle, know, absence and think played an important role in the network. Through this analysis, we identified the main keywords and their relationships within each topic and derived the main issues for each topic. This study confirmed the increasing and diversifying public interest and complaints on endangered wildlife and highlighted the need for professional response. We also suggested developing and extending participatory conservation plans that align with the public's preferences and demands. This study demonstrated the feasibility of using complaint big data on endangered wildlife and its implications for policy decision-making and public promotion on endangered wildlife.

Selection of Cluster Topic Words in Hierarchical Clustering using K-Means Algorithm

  • Lee Shin Won;Yi Sang Seon;An Dong Un;Chung Sung Jong
    • Proceedings of the IEEK Conference
    • /
    • 2004.08c
    • /
    • pp.885-889
    • /
    • 2004
  • Fast and high-quality document clustering algorithms play an important role in providing data exploration by organizing large amounts of information into a small number of meaningful clusters. Hierarchical clustering improves the performance of retrieval and makes that users can understand easily. For outperforming of clustering, we implemented hierarchical structure with variety and readability, by careful selection of cluster topic words and deciding the number of clusters dynamically. It is important to select topic words because hierarchical clustering structure is summarizes result of searching. We made choice of noun word as a cluster topic word. The quality of topic words is increased $33\%$ as follows. As the topic word of each cluster, the only noun word is extracted for the top-level cluster and the used topic words for the children clusters were not reused.

  • PDF

Exploring Topic Defining Patterns of Students in Interdisciplinary Capstone Design Class (캡스톤 디자인 수업에서 학생들의 주제 결정 패턴 탐색)

  • Byun, Moon Kyoung
    • Journal of Engineering Education Research
    • /
    • v.21 no.1
    • /
    • pp.14-26
    • /
    • 2018
  • The goal of this study was to explore topic defining patterns of students in interdisciplinary Capstone Design Class. Thematic analysis methodology was used to examine 85 Korean college students' lived experience of project topic generation which is for interdisciplinary capstone design class and Individual open-ended survey for constituted the data sources. Findings show four contexts of student's topic defining patterns using thematic analysis including (a) one leader's directed problem representation, (b) team common decision making after brainstorming, (c) empathy with professor proposed issue, (d) problems offered to students by corporate or research competitions. Based on research result, I could suggest instructional strategies of Capstone Design Class of teacher for helping their students' topic defining. It was necessary to minimize the opinions of the instructors at the beginning of class and minimize the number of team members. And also it provided a lot of opportunities to collaborate with companies in the topic selection process, it will help to develop the students' ability to determine the valuable topic in project.

A Design on Informal Big Data Topic Extraction System Based on Spark Framework (Spark 프레임워크 기반 비정형 빅데이터 토픽 추출 시스템 설계)

  • Park, Kiejin
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.11
    • /
    • pp.521-526
    • /
    • 2016
  • As on-line informal text data have massive in its volume and have unstructured characteristics in nature, there are limitations in applying traditional relational data model technologies for data storage and data analysis jobs. Moreover, using dynamically generating massive social data, social user's real-time reaction analysis tasks is hard to accomplish. In the paper, to capture easily the semantics of massive and informal on-line documents with unsupervised learning mechanism, we design and implement automatic topic extraction systems according to the mass of the words that consists a document. The input data set to the proposed system are generated first, using N-gram algorithm to build multiple words to capture the meaning of the sentences precisely, and Hadoop and Spark (In-memory distributed computing framework) are adopted to run topic model. In the experiment phases, TB level input data are processed for data preprocessing and proposed topic extraction steps are applied. We conclude that the proposed system shows good performance in extracting meaningful topics in time as the intermediate results come from main memories directly instead of an HDD reading.

Analysis of outdoor-wear research trends using topic modeling (토픽 모델링을 이용한 아웃도어웨어 연구 동향 분석)

  • Kihyang Han;Minsun Lee
    • The Research Journal of the Costume Culture
    • /
    • v.31 no.1
    • /
    • pp.53-69
    • /
    • 2023
  • This study aims to analyze research trends regarding outdoor wear. For this purpose, the data-collection period was limited to January 2002-October 2022, and the collection consisted of titles of papers, academic names, abstracts, and publication years from the Research Information Sharing Service (RISS). Frequency analysis was conducted on 227 papers in total to check academic journals and annual trends, and LDA topic-modeling analysis was conducted using 20,964 tokens. Data pre-processing was performed prior to topic-modeling analysis; after that, topic-modeling analysis, core topic derivation, and visualization were performed using a Python algorithm. A total of eight topics were obtained from the comprehensive analysis: experiential marketing and lifestyle, property and evaluation of outdoor wear, design and patterns of outdoor wear, outdoor-wear purchase behavior, color, designs and materials of outdoor wear, promotional strategies for outdoor wear, purchase intention and satisfaction depending on the brand image of outdoor wear, differences in outdoor wear preferences by consumer group. The results of topic-modeling analysis revealed that the topic, which includes a study on the design and material of outdoor wear and the pattern of jackets related to the overall shape, was the highest at 30.9% of the total topics. The next highest topic was also the design and color of outdoor wear, indicating that design-related research was the main research topic in outdoor wear research. It is hoped that analyzing outdoor wear research will help comprehend the research conducted thus far and reveal future directions.

Topic Extraction and Classification Method Based on Comment Sets

  • Tan, Xiaodong
    • Journal of Information Processing Systems
    • /
    • v.16 no.2
    • /
    • pp.329-342
    • /
    • 2020
  • In recent years, emotional text classification is one of the essential research contents in the field of natural language processing. It has been widely used in the sentiment analysis of commodities like hotels, and other commentary corpus. This paper proposes an improved W-LDA (weighted latent Dirichlet allocation) topic model to improve the shortcomings of traditional LDA topic models. In the process of the topic of word sampling and its word distribution expectation calculation of the Gibbs of the W-LDA topic model. An average weighted value is adopted to avoid topic-related words from being submerged by high-frequency words, to improve the distinction of the topic. It further integrates the highest classification of the algorithm of support vector machine based on the extracted high-quality document-topic distribution and topic-word vectors. Finally, an efficient integration method is constructed for the analysis and extraction of emotional words, topic distribution calculations, and sentiment classification. Through tests on real teaching evaluation data and test set of public comment set, the results show that the method proposed in the paper has distinct advantages compared with other two typical algorithms in terms of subject differentiation, classification precision, and F1-measure.

Topic Analysis of Scholarly Communication Research

  • Ji, Hyun;Cha, Mikyeong
    • Journal of Information Science Theory and Practice
    • /
    • v.9 no.2
    • /
    • pp.47-65
    • /
    • 2021
  • This study aims to identify specific topics, trends, and structural characteristics of scholarly communication research, based on 1,435 articles published from 1970 to 2018 in the Scopus database through Latent Dirichlet Allocation topic modeling, serial analysis, and network analysis. Topic modeling, time series analysis, and network analysis were used to analyze specific topics, trends, and structures, respectively. The results were summarized into three sets as follows. First, the specific topics of scholarly communication research were nineteen in number, including research resource management and research data, and their research proportion is even. Second, as a result of the time series analysis, there are three upward trending topics: Topic 6: Open Access Publishing, Topic 7: Green Open Access, Topic 19: Informal Communication, and two downward trending topics: Topic 11: Researcher Network and Topic 12: Electronic Journal. Third, the network analysis results indicated that high mean profile association topics were related to the institution, and topics with high triangle betweenness centrality, such as Topic 14: Research Resource Management, shared the citation context. Also, through cluster analysis using parallel nearest neighbor clustering, six clusters connected with different concepts were identified.

Analysis of Laughter Therapy Trend Using Text Network Analysis and Topic Modeling

  • LEE, Do-Young
    • Journal of Wellbeing Management and Applied Psychology
    • /
    • v.5 no.4
    • /
    • pp.33-37
    • /
    • 2022
  • Purpose: This study aims to understand the trend and central concept of domestic researches on laughter therapy. For the analysis, this study used total 72 theses verified by inputting the keyword 'laughter therapy' from 2007 to 2021. Research design, data and methodology: This study performed the development and analysis of keyword co-occurrence network, analyzed the types of researches through topic modeling, and verified the visualized word cloud and sociogram. The keyword data that was cleaned through preprocessing, was analyzed in the method of centrality analysis and topic modeling through the 1-mode matrix conversion process by using the NetMiner (version 4.4) Program. Results: The keywords that most appeared for last 14 years were laughter therapy, depression, the elderly, and stress. The five topics analyzed in thesis data from 2007 to 2021 were therapy, cognitive behavior, quality of life, stress, and the elderly. Conclusions: This study understood the flow and trend of research topics of domestic laughter therapy for last 14 years, and there should be continuous researches on laughter therapy, which reflects the flow of time in the future.

A Study on the Document Topic Extraction System Based on Big Data (빅데이터 기반 문서 토픽 추출 시스템 연구)

  • Hwang, Seung-Yeon;An, Yoon-Bin;Shin, Dong-Jin;Oh, Jae-Kon;Moon, Jin Yong;Kim, Jeong-Joon
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.20 no.5
    • /
    • pp.207-214
    • /
    • 2020
  • Nowadays, the use of smart phones and various electronic devices is increasing, the Internet and SNS are activated, and we live in the flood of information. The amount of information has grown exponentially, making it difficult to look at a lot of information, and more and more people want to see only key keywords in a document, and the importance of research to extract topics that are the core of information is increasing. In addition, it is also an important issue to extract the topic and compare it with the past to infer the current trend. Topic modeling techniques can be used to extract topics from a large volume of documents, and these extracted topics can be used in various fields such as trend prediction and data analysis. In this paper, we inquire the topic of the three-year papers of 2016, 2017, and 2018 in the field of computing using the LDA algorithm, one of Probabilistic Topic Model Techniques, in order to analyze the rapidly changing trends and keep pace with the times. Then we analyze trends and flows of research.