• Title/Summary/Keyword: Text data

Search Result 2,956, Processing Time 0.029 seconds

Enhancing Multimodal Emotion Recognition in Speech and Text with Integrated CNN, LSTM, and BERT Models (통합 CNN, LSTM, 및 BERT 모델 기반의 음성 및 텍스트 다중 모달 감정 인식 연구)

  • Edward Dwijayanto Cahyadi;Hans Nathaniel Hadi Soesilo;Mi-Hwa Song
    • The Journal of the Convergence on Culture Technology
    • /
    • v.10 no.1
    • /
    • pp.617-623
    • /
    • 2024
  • Identifying emotions through speech poses a significant challenge due to the complex relationship between language and emotions. Our paper aims to take on this challenge by employing feature engineering to identify emotions in speech through a multimodal classification task involving both speech and text data. We evaluated two classifiers-Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM)-both integrated with a BERT-based pre-trained model. Our assessment covers various performance metrics (accuracy, F-score, precision, and recall) across different experimental setups). The findings highlight the impressive proficiency of two models in accurately discerning emotions from both text and speech data.

Advancing Defect Resolution in Construction: Integrating Text Mining and Semantic Analysis for Deeper Customer Experiences

  • Wonwoo Shin;SangHyeok Han;Sungkon Moon
    • International conference on construction engineering and project management
    • /
    • 2024.07a
    • /
    • pp.689-697
    • /
    • 2024
  • According to the South Korean Ministry of Land, Infrastructure, and Transport, instances of defect dispute resolutions, primarily between construction contractors and apartment occupants, have been occurring at an annual average of over 4,000 cases since 2014 to the present day. To address the persistent issue of disputes between contractors and occupants regarding construction defects, it is crucial to use customer sentiment analysis to improve customer rights and guide construction companies in their efforts. This study presents a methodology for effectively managing customer complaints and enhancing feedback analysis in the context of defect repair services. The study begins with collecting and preprocessing customer feedback data. Semantic network analysis is used to understand the causes of discomfort in customer feedback, revealing insights into the emotional sentiments expressed by customers and identifying causal relationships between emotions and themes. This research combines text mining, and semantic network analysis to analyze customer feedback for decision-making. By doing so, defect repair service providers can improve service quality, address customer concerns promptly, and understand the factors behind emotional responses in customer feedback. Through data-driven decision-making, these providers can enhance customer rights and identify areas for construction companies to improve service quality.

Analyzing Trend of Clinic-Level Medical Institutions and Pharmacy Name Using Text Mining Technique (텍스트마이닝을 통한 의원급 의료기관 및 약국 명칭 유행 분석)

  • Inseong Lee;NamKwen Kim
    • The Journal of Korean Medicine
    • /
    • v.45 no.3
    • /
    • pp.122-130
    • /
    • 2024
  • Objectives: This study aimed to analyze trend of names of clinic-level medical institutions and pharmacy in Korea to understand interaction between health care provider and consumer. Methods: This study analyzes 60,101 medical clinic names, 27,609 dental clinic names, 25,506 Korean medicine clinic names, and 68,032 pharmacy names. Official records from Korea Local Information Research & Development Institute, data from 1946 to June 2024. Researcher uses text-mining technique including tokenizer of Soynlp and wordcloud to visualize trends. Results: In pharmacy business names, franchise name 'Onnuri' is dominant. In case of dental clinic and Korean medicine clinic, university names like 'Seoul', 'Yeonsei', 'Kyunghee' are dominant. Similarly, medical clinic names include university names but also teaching hospital names like 'Samsung' or 'Seongmo(St. Mary)'. Commonly used word in clinic-level medical institution and pharmacy is 'Uri' and latest trend words are 'Bareun' and '365'. Conclusions: Methods presented in this study for analyzing business names can directly applied to the study of business names in other industries. The results of this study would develop marketing strategies of medical industry. Furthermore, since business name provide concentrated data on market perception and the interaction between consumers and providers, they can be useful for sociology and medical humanities researchers.

Keyword Network Visualization for Text Summarization and Comparative Analysis (문서 요약 및 비교분석을 위한 주제어 네트워크 가시화)

  • Kim, Kyeong-rim;Lee, Da-yeong;Cho, Hwan-Gue
    • Journal of KIISE
    • /
    • v.44 no.2
    • /
    • pp.139-147
    • /
    • 2017
  • Most of the information prevailing in the Internet space consists of textual information. So one of the main topics regarding the huge document analyses that are required in the "big data" era is the development of an automated understanding system for textual data; accordingly, the automation of the keyword extraction for text summarization and abstraction is a typical research problem. But the simple listing of a few keywords is insufficient to reveal the complex semantic structures of the general texts. In this paper, a text-visualization method that constructs a graph by computing the related degrees from the selected keywords of the target text is developed; therefore, two construction models that provide the edge relation are proposed for the computing of the relation degree among keywords, as follows: influence-interval model and word- distance model. The finally visualized graph from the keyword-derived edge relation is more flexible and useful for the display of the meaning structure of the target text; furthermore, this abstract graph enables a fast and easy understanding of the target text. The authors' experiment showed that the proposed abstract-graph model is superior to the keyword list for the attainment of a semantic and comparitive understanding of text.

Financial Fraud Detection using Text Mining Analysis against Municipal Cybercriminality (지자체 사이버 공간 안전을 위한 금융사기 탐지 텍스트 마이닝 방법)

  • Choi, Sukjae;Lee, Jungwon;Kwon, Ohbyung
    • Journal of Intelligence and Information Systems
    • /
    • v.23 no.3
    • /
    • pp.119-138
    • /
    • 2017
  • Recently, SNS has become an important channel for marketing as well as personal communication. However, cybercrime has also evolved with the development of information and communication technology, and illegal advertising is distributed to SNS in large quantity. As a result, personal information is lost and even monetary damages occur more frequently. In this study, we propose a method to analyze which sentences and documents, which have been sent to the SNS, are related to financial fraud. First of all, as a conceptual framework, we developed a matrix of conceptual characteristics of cybercriminality on SNS and emergency management. We also suggested emergency management process which consists of Pre-Cybercriminality (e.g. risk identification) and Post-Cybercriminality steps. Among those we focused on risk identification in this paper. The main process consists of data collection, preprocessing and analysis. First, we selected two words 'daechul(loan)' and 'sachae(private loan)' as seed words and collected data with this word from SNS such as twitter. The collected data are given to the two researchers to decide whether they are related to the cybercriminality, particularly financial fraud, or not. Then we selected some of them as keywords if the vocabularies are related to the nominals and symbols. With the selected keywords, we searched and collected data from web materials such as twitter, news, blog, and more than 820,000 articles collected. The collected articles were refined through preprocessing and made into learning data. The preprocessing process is divided into performing morphological analysis step, removing stop words step, and selecting valid part-of-speech step. In the morphological analysis step, a complex sentence is transformed into some morpheme units to enable mechanical analysis. In the removing stop words step, non-lexical elements such as numbers, punctuation marks, and double spaces are removed from the text. In the step of selecting valid part-of-speech, only two kinds of nouns and symbols are considered. Since nouns could refer to things, the intent of message is expressed better than the other part-of-speech. Moreover, the more illegal the text is, the more frequently symbols are used. The selected data is given 'legal' or 'illegal'. To make the selected data as learning data through the preprocessing process, it is necessary to classify whether each data is legitimate or not. The processed data is then converted into Corpus type and Document-Term Matrix. Finally, the two types of 'legal' and 'illegal' files were mixed and randomly divided into learning data set and test data set. In this study, we set the learning data as 70% and the test data as 30%. SVM was used as the discrimination algorithm. Since SVM requires gamma and cost values as the main parameters, we set gamma as 0.5 and cost as 10, based on the optimal value function. The cost is set higher than general cases. To show the feasibility of the idea proposed in this paper, we compared the proposed method with MLE (Maximum Likelihood Estimation), Term Frequency, and Collective Intelligence method. Overall accuracy and was used as the metric. As a result, the overall accuracy of the proposed method was 92.41% of illegal loan advertisement and 77.75% of illegal visit sales, which is apparently superior to that of the Term Frequency, MLE, etc. Hence, the result suggests that the proposed method is valid and usable practically. In this paper, we propose a framework for crisis management caused by abnormalities of unstructured data sources such as SNS. We hope this study will contribute to the academia by identifying what to consider when applying the SVM-like discrimination algorithm to text analysis. Moreover, the study will also contribute to the practitioners in the field of brand management and opinion mining.

Investigating Dynamic Mutation Process of Issues Using Unstructured Text Analysis (비정형 텍스트 분석을 활용한 이슈의 동적 변이과정 고찰)

  • Lim, Myungsu;Kim, Namgyu
    • Journal of Intelligence and Information Systems
    • /
    • v.22 no.1
    • /
    • pp.1-18
    • /
    • 2016
  • Owing to the extensive use of Web media and the development of the IT industry, a large amount of data has been generated, shared, and stored. Nowadays, various types of unstructured data such as image, sound, video, and text are distributed through Web media. Therefore, many attempts have been made in recent years to discover new value through an analysis of these unstructured data. Among these types of unstructured data, text is recognized as the most representative method for users to express and share their opinions on the Web. In this sense, demand for obtaining new insights through text analysis is steadily increasing. Accordingly, text mining is increasingly being used for different purposes in various fields. In particular, issue tracking is being widely studied not only in the academic world but also in industries because it can be used to extract various issues from text such as news, (SocialNetworkServices) to analyze the trends of these issues. Conventionally, issue tracking is used to identify major issues sustained over a long period of time through topic modeling and to analyze the detailed distribution of documents involved in each issue. However, because conventional issue tracking assumes that the content composing each issue does not change throughout the entire tracking period, it cannot represent the dynamic mutation process of detailed issues that can be created, merged, divided, and deleted between these periods. Moreover, because only keywords that appear consistently throughout the entire period can be derived as issue keywords, concrete issue keywords such as "nuclear test" and "separated families" may be concealed by more general issue keywords such as "North Korea" in an analysis over a long period of time. This implies that many meaningful but short-lived issues cannot be discovered by conventional issue tracking. Note that detailed keywords are preferable to general keywords because the former can be clues for providing actionable strategies. To overcome these limitations, we performed an independent analysis on the documents of each detailed period. We generated an issue flow diagram based on the similarity of each issue between two consecutive periods. The issue transition pattern among categories was analyzed by using the category information of each document. In this study, we then applied the proposed methodology to a real case of 53,739 news articles. We derived an issue flow diagram from the articles. We then proposed the following useful application scenarios for the issue flow diagram presented in the experiment section. First, we can identify an issue that actively appears during a certain period and promptly disappears in the next period. Second, the preceding and following issues of a particular issue can be easily discovered from the issue flow diagram. This implies that our methodology can be used to discover the association between inter-period issues. Finally, an interesting pattern of one-way and two-way transitions was discovered by analyzing the transition patterns of issues through category analysis. Thus, we discovered that a pair of mutually similar categories induces two-way transitions. In contrast, one-way transitions can be recognized as an indicator that issues in a certain category tend to be influenced by other issues in another category. For practical application of the proposed methodology, high-quality word and stop word dictionaries need to be constructed. In addition, not only the number of documents but also additional meta-information such as the read counts, written time, and comments of documents should be analyzed. A rigorous performance evaluation or validation of the proposed methodology should be performed in future works.

Dynamic Compressed Representation of Texts with Rank/Select

  • Lee, Sun-Ho;Park, Kun-Soo
    • Journal of Computing Science and Engineering
    • /
    • v.3 no.1
    • /
    • pp.15-26
    • /
    • 2009
  • Given an n-length text T over a $\sigma$-size alphabet, we present a compressed representation of T which supports retrieving queries of rank/select/access and updating queries of insert/delete. For a measure of compression, we use the empirical entropy H(T), which defines a lower bound nH(T) bits for any algorithm to compress T of n log $\sigma$ bits. Our representation takes this entropy bound of T, i.e., nH(T) $\leq$ n log $\sigma$ bits, and an additional bits less than the text size, i.e., o(n log $\sigma$) + O(n) bits. In compressed space of nH(T) + o(n log $\sigma$) + O(n) bits, our representation supports O(log n) time queries for a log n-size alphabet and its extension provides O(($1+\frac{{\log}\;{\sigma}}{{\log}\;{\log}\;n}$) log n) time queries for a $\sigma$-size alphabet.

Automatic Text Categorization Using Hybrid Multiple Model Schemes (하이브리드 다중모델 학습기법을 이용한 자동 문서 분류)

  • 명순희;김인철
    • Journal of the Korean Society for information Management
    • /
    • v.19 no.4
    • /
    • pp.35-51
    • /
    • 2002
  • Inductive learning and classification techniques have been employed in various research and applications that organize textual data to solve the problem of information access. In this study, we develop hybrid model combination methods which incorporate the concepts and techniques for multiple modeling algorithms to improve the accuracy of text classification, and conduct experiments to evaluate the performances of proposed schemes. Boosted stacking, one of the extended stacking schemes proposed in this study yields higher accuracy relative to the conventional model combination methods and single classifiers.

Recent Development in Text-based Medical Image Retrieval (텍스트 기반 의료영상 검색의 최근 발전)

  • Hwang, Kyung Hoon;Lee, Haejun;Koh, Geon;Kim, Seog Gyun;Sun, Yong Han;Choi, Duckjoo
    • Journal of Biomedical Engineering Research
    • /
    • v.36 no.3
    • /
    • pp.55-60
    • /
    • 2015
  • An effective image retrieval system is required as the amount of medical imaging data is increasing recently. Authors reviewed the recent development of text-based medical image retrieval including the use of controlled vocabularies - RadLex (Radiology Lexicon), FMA (Foundational Model of Anatomy), etc - natural language processing, semantic ontology, and image annotation and markup.

Using Text Mining Techniques for Intrusion Detection Problem in Computer Network (텍스트 마이닝 기법을 이용한 컴퓨터 네트워크의 침입 탐지)

  • Oh Seung-Joon;Won Min-Kwon
    • Journal of the Korea Society of Computer and Information
    • /
    • v.10 no.5 s.37
    • /
    • pp.27-32
    • /
    • 2005
  • Recently there has been much interest in applying data mining to computer network intrusion detection. A new approach, based on the k-Nearest Neighbour(kNN) classifier, is used to classify Program behaviour as normal or intrusive. Each system call is treated as a word and the collection of system calls over each program execution as a document. These documents are then classified using kNN classifier, a Popular method in text mining. A simple example illustrates the proposed procedure.

  • PDF