• Title/Summary/Keyword: latent dirichlet allocation (LDA)

Search Result 182, Processing Time 0.021 seconds

An Examination of the Topics and Changes in the Research Papers Published in the Journal of Korean Elementary Science Education Using Latent Dirichlet Allocation for the Topic Modeling Analysis (잠재 디리클레 할당(LDA) 기반의 토픽모델링 분석을 통한 '초등과학교육' 학술지 연구논문의 주제 및 변화)

  • Chang, Jina;Na, Jiyeon
    • Journal of Korean Elementary Science Education
    • /
    • v.41 no.2
    • /
    • pp.356-372
    • /
    • 2022
  • This study examined the topics that have appeared in the "Journal of Korean Elementary Science Education" over the past 50 years to identify the changes that have occurred in the Korean Society of Elementary Science Education. Latent Dirichlet allocation topic modeling was applied to 1,065 English abstracts from the first issue (1983) to 2021, from which 14 main topics were extracted. The meaning of each topic was then analyzed from its keywords and documents. Subsequently, to elucidate the topic trends, the topics' increase or decrease every three years was statistically examined through linear regression analysis. Based on the results, implications for developing and supporting elementary science education research in the future were discussed.

Trend Analysis of School Health Research using Latent Semantic Analysis (잠재의미분석방법을 통한 학교보건 연구동향 분석)

  • Shin, Seon-Hi;Park, Youn-Ju
    • Journal of the Korean Society of School Health
    • /
    • v.33 no.3
    • /
    • pp.184-193
    • /
    • 2020
  • Purpose: This study was designed to investigate the trends in school health research in Korea using probabilistic latent semantic analysis. The study longitudinally analyzed the abstracts of the papers published in 「The Journal of the Korean Society of School Health」 over the recent 17 years, which is between 2004 and August 2020. By classifying all the papers according to the topics identified through the analysis, it was possible to see how the distribution of the topics has changed over years. Based on the results, implications for school health research and educational uses of latent semantic analysis were suggested. Methods: This study investigated the research trends by longitudinally analyzing journal abstracts using latent dirichlet allocation (LDA), a type of LSA. The abstracts in 「The Journal of the Korean Society of School Health」 published from 2004 to August 2020 were used for the analysis. Results: A total of 34 latent topics were identified by LDA. Six topics, which were「Adolescent depression and suicide prevention」, 「Students' knowledge, attitudes, & behaviors」, 「Effective self-esteem program through depression interventions」, 「Factors of students' stress」, 「Intervention program to prevent adolescent risky behaviors」, and 「Sex education curriculum, and teacher」were most frequently covered by the journal. Each of them was dealt with in at least 20 papers. The topics related to 「Intervention program to prevent adolescent risky behaviors」, 「Effective self-esteem program through depression interventions」, and 「Preventive vaccination and factors of effective vaccination」 appeared repeatedly over the most recent 5 years. Conclusion: This study introduced an AI-powered analysis method that enables data-centered objective text analysis without human intervention. Based on the results, implications for school health research were presented, and various uses of latent semantic analysis (LSA) in educational research were suggested.

Latent mobility pattern analysis of bus passengers with LDA (LDA 기법을 이용한 버스 승객의 잠재적 이동패턴 분석)

  • Cho, Ah;Lee, Kyung Hee;Cho, Wan Sup
    • Journal of the Korean Data and Information Science Society
    • /
    • v.26 no.5
    • /
    • pp.1061-1069
    • /
    • 2015
  • Recently, transportation big data generated in the transportation sector has been widely used in the transportation policies making and efficient system management. Bus passengers' mobility patterns are useful insight for transportation policy maker to optimize bus lines and time intervals in a city. We propose a new methodology to discover mobility patterns by using transportation card data. We first estimate the bus stations where the passengers get-off because the transportation card data don't have the get-off information in most cities. We then applies LDA (Latent Dirichlet Allocation), the most representative topic modeling technique, to discover mobility patterns of bus passengers in Cheong-Ju city. To understand discovered patterns, we construct a data warehouse and perform multi-dimensional analysis by bus-route, region, time-period, and the mobility patterns (get-on/get-off station). In the case of Cheong Ju, we discovered mobility pattern 1 from suburban area to Cheong-Ju terminal, mobility pattern 2 from residential area to commercial area, mobility pattern 3 from school areas to commercial area.

Systemic Analysis of Research Activities and Trends Related to Artificial Intelligence(A.I.) Technology Based on Latent Dirichlet Allocation (LDA) Model (Latent Dirichlet Allocation (LDA) 모델 기반의 인공지능(A.I.) 기술 관련 연구 활동 및 동향 분석)

  • Chung, Myoung Sug;Lee, Joo Yeoun
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.23 no.3
    • /
    • pp.87-95
    • /
    • 2018
  • Recently, with the technological development of artificial intelligence, related market is expanding rapidly. In the artificial intelligence technology field, which is still in the early stage but still expanding, it is important to reduce uncertainty about research direction and investment field. Therefore, this study examined technology trends using text mining and topic modeling among big data analysis methods and suggested trends of core technology and future growth potential. We hope that the results of this study will provide researchers with an understanding of artificial intelligence technology trends and new implications for future research directions.

KOSPI index prediction using topic modeling and LSTM

  • Jin-Hyeon Joo;Geun-Duk Park
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.7
    • /
    • pp.73-80
    • /
    • 2024
  • In this paper, we proposes a method to improve the accuracy of predicting the Korea Composite Stock Price Index (KOSPI) by combining topic modeling and Long Short-Term Memory (LSTM) neural networks. In this paper, we use the Latent Dirichlet Allocation (LDA) technique to extract ten major topics related to interest rate increases and decreases from financial news data. The extracted topics, along with historical KOSPI index data, are input into an LSTM model to predict the KOSPI index. The proposed model has the characteristic of predicting the KOSPI index by combining the time series prediction method by inputting the historical KOSPI index into the LSTM model and the topic modeling method by inputting news data. To verify the performance of the proposed model, this paper designs four models (LSTM_K model, LSTM_KNS model, LDA_K model, LDA_KNS model) based on the types of input data for the LSTM and presents the predictive performance of each model. The comparison of prediction performance results shows that the LSTM model (LDA_K model), which uses financial news topic data and historical KOSPI index data as inputs, recorded the lowest RMSE (Root Mean Square Error), demonstrating the best predictive performance.

Comparison and Analysis of Subject Classification for Domestic Research Data (국내 학술논문 주제 분류 알고리즘 비교 및 분석)

  • Choi, Wonjun;Sul, Jaewook;Jeong, Heeseok;Yoon, Hwamook
    • The Journal of the Korea Contents Association
    • /
    • v.18 no.8
    • /
    • pp.178-186
    • /
    • 2018
  • Subject classification of thesis units is essential to serve scholarly information deliverables. However, to date, there is a journal-based topic classification, and there are not many article-level subject classification services. In the case of academic papers among domestic works, subject classification can be a more important information because it can cover a larger area of service and can provide service by setting a range. However, the problem of classifying themes by field requires the hands of experts in various fields, and various methods of verification are needed to increase accuracy. In this paper, we try to classify topics using the unsupervised learning algorithm to find the correct answer in the unknown state and compare the results of the subject classification algorithms using the coherence and perplexity. The unsupervised learning algorithms are a well-known Hierarchical Dirichlet Process (HDP), Latent Dirichlet Allocation (LDA) and Latent Semantic Indexing (LSI) algorithm.

Feature Expansion based on LDA Word Distribution for Performance Improvement of Informal Document Classification (비격식 문서 분류 성능 개선을 위한 LDA 단어 분포 기반의 자질 확장)

  • Lee, Hokyung;Yang, Seon;Ko, Youngjoong
    • Journal of KIISE
    • /
    • v.43 no.9
    • /
    • pp.1008-1014
    • /
    • 2016
  • Data such as Twitter, Facebook, and customer reviews belong to the informal document group, whereas, newspapers that have grammar correction step belong to the formal document group. Finding consistent rules or patterns in informal documents is difficult, as compared to formal documents. Hence, there is a need for additional approaches to improve informal document analysis. In this study, we classified Twitter data, a representative informal document, into ten categories. To improve performance, we revised and expanded features based on LDA(Latent Dirichlet allocation) word distribution. Using LDA top-ranked words, the other words were separated or bundled, and the feature set was thus expanded repeatedly. Finally, we conducted document classification with the expanded features. Experimental results indicated that the proposed method improved the micro-averaged F1-score of 7.11%p, as compared to the results before the feature expansion step.

Classifying and Characterizing the Types of Gentrified Commercial Districts Based on Sense of Place Using Big Data: Focusing on 14 Districts in Seoul (빅데이터를 활용한 젠트리피케이션 상권의 장소성 분류와 특성 분석 -서울시 14개 주요상권을 중심으로-)

  • Young-Jae Kim;In Kwon Park
    • Journal of the Korean Regional Science Association
    • /
    • v.39 no.1
    • /
    • pp.3-20
    • /
    • 2023
  • This study aims to categorize the 14 major gentrified commercial areas of Seoul and analyze their characteristics based on their sense of place. To achieve this, we conducted hierarchical cluster analysis using text data collected from Naver Blog. We divided the districts into two dimensions: "experience" and "feature" and analyzed their characteristics using LDA (Latent Dirichlet Allocation) of the text data and statistical data collected from Seoul Open Data Square. As a result, we classified the commercial districts of Seoul into 5 categories: 'theater district,' 'traditional cultural district,' 'female-beauty district,' 'exclusive restaurant and medical district,' and 'trend-leading district.' The findings of this study are expected to provide valuable insights for policy-makers to develop more efficient and suitable commercial policies.

Text Analytics for Classifying Types of Accident Occurrence Using Accident Report Documents (사고보고문서를 이용한 텍스트 기반 사고발생 유형 및 관계 분석)

  • Kim, Beom Soo;Chang, Seongrok;Suh, Yongyoon
    • Journal of the Korean Society of Safety
    • /
    • v.33 no.3
    • /
    • pp.58-64
    • /
    • 2018
  • Recently, a lot of accident report documents have accumulated in almost all of industries, including critical information of accidents. Accordingly, text data contained in accident report documents are considered useful information for understanding accident processes. However, there has been a lack of systematic approaches to analyzing accident report documents. In this respect, this paper aims at proposing text analytics approach to extracting critical information on accident processes. To be specific, major causes of the accident occurrence are classified based on text information contained in accident report documents by using both textmining and latent Dirichlet allocation (LDA) algorithms. The textmining algorithm is used to structure the document-term matrix and the LDA algorithm is applied to extract latent topics included in a lot of accident report documents. We extract ten topics of accidents as accident types and related keywords of accidents with respect to each accident type. The cause-and-effect diagram is then depicted as a tool for navigating processes of the accident occurrence by structuring causes extracted from LDA. Further, the trends of accidents are identified to explore patterns of accident occurrence in each of types. Three patterns of increasing to decreasing, decreasing to increasing, or only increasing are presented in the case of a chemical plant. The proposed approach helps safety managers systematically supervise the causes and processes of accidents through analysis of text information contained in accident report documents.

Establishment of ITS Policy Issues Investigation Method in the Road Section applied Textmining (텍스트마이닝을 활용한 도로분야 ITS 정책이슈 탐색기법 정립)

  • Oh, Chang-Seok;Lee, Yong-taeck;Ko, Minsu
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.15 no.6
    • /
    • pp.10-23
    • /
    • 2016
  • With requiring circumspections using big data, this study attempts to develop and apply the search method for audit issues relating to the ITS policy or program. For the foregoing, the auditing process of the board of audit and inspection was converged with the theoretical frame of boundary analysis proposed by William Dunn as an analysis tool for audit issues. Moreover, we apply the text mining technique in order to computerize the analysis tool, which is similar to the boundary analysis in the concept of approaching meta-problems. For the text mining analysis, specific model we applied the antisymmetry-symmetry compound lexeme-based LDA model based on the Latent Dirichlet Allocation(LDA) methodologies proposed by David Blei. The several prime issues were founded through a case analysis as follows: lack of collection of traffic information by the urban traffic information system, which is operated by the National Police Agency, the overlapping problems between the Ministry of Land, Infrastructure and Transport and the Advanced Traffic Management System and fabrication of the mileage on digital tachograph.