• Title/Summary/Keyword: Text clustering

Search Result 205, Processing Time 0.033 seconds

A Novel Technique of Topic Detection for On-line Text Documents: A Topic Tree-based Approach (온라인 텍스트문서의 계층적 트리 기반 주제탐색 기법)

  • Xuan, Man;Kim, Han-Joon
    • Proceedings of the Korea Information Processing Society Conference
    • /
    • 2012.11a
    • /
    • pp.396-399
    • /
    • 2012
  • Topic detection is a problem of discovering the topics of online publishing documents. For topic detection, it is important to extract correct topic words and to show the topical words easily to understand. We consider a topic tree-based approach to more effectively and more briefly show the result of topic detection for online text documents. In this paper, to achieve the topic tree-based topic detection, we propose a new term weighting method, called CTF-CDF-IDF, which is simple yet effective. Moreover, we have modified a conventional clustering method, which we call incremental k-medoids algorithm. Our experimental results with Reuters-21578 and Google news collections show that the proposed method is very useful for topic detection.

Text Clustering Algorithm Based on Ontology Concepts Combination (온톨로지 개념 합병 기반 문서 군집화 기법)

  • Guan, XiangDong;Kim, Woosaeng
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2012.10a
    • /
    • pp.722-724
    • /
    • 2012
  • 문서 군집화를 통하여 문서를 효율적으로 조직, 관리, 검색 할 수 있다. 일반적으로 문서 군집화는 많은 단어와 개념들을 포함하고 있기 때문에 차원이 큰 벡터 공간 모델에서 군집화를 수행한다. 본 논문에서 문서 집합에 대응하는 온톨로지를 이용하여 문서 벡터 공간의 차원을 줄여 효율적으로 군집화하는 방법을 제안하고, 실험을 통하여 기존 방법보다 우수함을 보인다.

  • PDF

Automatic Text Summarization with Lexical Clustering (어휘 클러스터링을 이용한 자동 문서 요약)

  • 김건오;고영중;서정연
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2002.04b
    • /
    • pp.463-465
    • /
    • 2002
  • 자동 문서 요약 시스템은 문서내 담겨있는 정보를 최대만 표현하면서 문서의 크기를 줄이는 시스템이다. 본 논문에서는 어휘를 자동으로 클러스터링하여 문서 대표어를 찾고, 이를 제목과 조합하여 요약을 수행하는 시스템을 제안한다. 특히 이 시스템은 제목이 없는 문서도 요약을 수행할 수 있는 장점이 있다. 비교시스템으로는 제목, 위치, 빈도를 이용만 시스템을 구축하여 사용하였으며 30%, 10%, 그리고 4문장 요약에서 제안한 시스템은 모두 우수한 성능을 보였다.

  • PDF

Perceptions and Trends of Digital Fashion Technology - A Big Data Analysis - (빅데이터 분석을 이용한 디지털 패션 테크에 대한 인식 연구)

  • Song, Eun-young;Lim, Ho-sun
    • Fashion & Textile Research Journal
    • /
    • v.23 no.3
    • /
    • pp.380-389
    • /
    • 2021
  • This study aimed to reveal the perceptions and trends of digital fashion technology through an informational approach. A big data analysis was conducted after collecting the text shown in a web environment from April 2019 to April 2021. Key words were derived through text mining analysis and network analysis, and the structure of perception of digital fashion technology was identified. Using textoms, we collected 8144 texts after data refinement, conducted a frequency of emergence and central component analysis, and visualized the results with word cloud and N-gram. The frequency of appearance also generated matrices with the top 70 words, and a structural equivalent analysis was performed. The results were presented with network visualizations and dendrograms. Fashion, digital, and technology were the most frequently mentioned topics, and the frequencies of platform, digital transformation, and start-ups were also high. Through clustering, four clusters of marketing were formed using fashion, digital technology, startups, and augmented reality/virtual reality technology. Future research on startups and smart factories with technologies based on stable platforms is needed. The results of this study contribute to increasing the fashion industry's knowledge on digital fashion technology and can be used as a foundational study for the development of research on related topics.

Research Trend on Diabetes Mobile Applications: Text Network Analysis and Topic Modeling (당뇨병 모바일 앱 관련 연구동향: 텍스트 네트워크 분석 및 토픽 모델링)

  • Park, Seungmi;Kwak, Eunju;Kim, Youngji
    • Journal of Korean Biological Nursing Science
    • /
    • v.23 no.3
    • /
    • pp.170-179
    • /
    • 2021
  • Purpose: The aim of this study was to identify core keywords and topic groups in the 'Diabetes mellitus and mobile applications' field of research for better understanding research trends in the past 20 years. Methods: This study was a text-mining and topic modeling study including four steps such as 'collecting abstracts', 'extracting and cleaning semantic morphemes', 'building a co-occurrence matrix', and 'analyzing network features and clustering topic groups'. Results: A total of 789 papers published between 2002 and 2021 were found in databases (Springer). Among them, 435 words were extracted from 118 articles selected according to the conditions: 'analyzed by text network analysis and topic modeling'. The core keywords were 'self-management', 'intervention', 'health', 'support', 'technique' and 'system'. Through the topic modeling analysis, four themes were derived: 'intervention', 'blood glucose level control', 'self-management' and 'mobile health'. The main topic of this study was 'self-management'. Conclusion: While more recent work has investigated mobile applications, the highest feature was related to self-management in the diabetes care and prevention. Nursing interventions utilizing mobile application are expected to not only effective and powerful glycemic control and self-management tools, but can be also used for patient-driven lifestyle modification.

A study on searching image by cluster indexing and sequential I/O (연속적 I/O와 클러스터 인덱싱 구조를 이용한 이미지 데이타 검색 연구)

  • Kim, Jin-Ok;Hwang, Dae-Joon
    • The KIPS Transactions:PartD
    • /
    • v.9D no.5
    • /
    • pp.779-788
    • /
    • 2002
  • There are many technically difficult issues in searching multimedia data such as image, video and audio because they are massive and more complex than simple text-based data. As a method of searching multimedia data, a similarity retrieval has been studied to retrieve automatically basic features of multimedia data and to make a search among data with retrieved features because exact match is not adaptable to a matrix of features of multimedia. In this paper, data clustering and its indexing are proposed as a speedy similarity-retrieval method of multimedia data. This approach clusters similar images on adjacent disk cylinders and then builds Indexes to access the clusters. To minimize the search cost, the hashing is adapted to index cluster. In addition, to reduce I/O time, the proposed searching takes just one I/O to look up the location of the cluster containing similar object and one sequential file I/O to read in this cluster. The proposed schema solves the problem of multi-dimension by using clustering and its indexing and has higher search efficiency than the content-based image retrieval that uses only clustering or indexing structure.

Similarity Analysis of Hospitalization using Crowding Distance

  • Jung, Yong Gyu;Choi, Young Jin;Cha, Byeong Heon
    • International journal of advanced smart convergence
    • /
    • v.5 no.2
    • /
    • pp.53-58
    • /
    • 2016
  • With the growing use of big data and data mining, it serves to understand how such techniques can be used to understand various relationships in the healthcare field. This study uses hierarchical methods of data analysis to explore similarities in hospitalization across several New York state counties. The study utilized methods of measuring crowding distance of data for age-specific hospitalization period. Crowding distance is defined as the longest distance, or least similarity, between urban cities. It is expected that the city of Clinton have the greatest distance, while Albany the other cities are closer because they are connected by the shortest distance to each step. Similarities were stronger across hospital stays categorized by age. Hierarchical clustering can be applied to predict the similarity of data across the 10 cities of hospitalization with the measurement of crowding distance. In order to enhance the performance of hierarchical clustering, comparison can be made across congestion distance when crowding distance is applied first through the application of converting text to an attribute vector. Measurements of similarity between two objects are dependent on the measurement method used in clustering but is distinguished from the similarity of the distance; where the smaller the distance value the more similar two things are to one other. By applying this specific technique, it is found that the distance between crowding is reduced consistently in relationship to similarity between the data increases to enhance the performance of the experiments through the application of special techniques. Furthermore, through the similarity by city hospitalization period, when the construction of hospital wards in cities, by referring to results of experiments, or predict possible will land to the extent of the size of the hospital facilities hospital stay is expected to be useful in efficiently managing the patient in a similar area.

Corrupted Region Restoration based on 2D Tensor Voting (2D 텐서 보팅에 기반 한 손상된 텍스트 영상의 복원 및 분할)

  • Park, Jong-Hyun;Toan, Nguyen Dinh;Lee, Guee-Sang
    • The KIPS Transactions:PartB
    • /
    • v.15B no.3
    • /
    • pp.205-210
    • /
    • 2008
  • A new approach is proposed for restoration of corrupted regions and segmentation in natural text images. The challenge is to fill in the corrupted regions on the basis of color feature analysis by second order symmetric stick tensor. It is show how feature analysis can benefit from analyzing features using tensor voting with chromatic and achromatic components. The proposed method is applied to text images corrupted by manifold types of various noises. Firstly, we decompose an image into chromatic and achromatic components to analyze images. Secondly, selected feature vectors are analyzed by second-order symmetric stick tensor. And tensors are redefined by voting information with neighbor voters, while restore the corrupted regions. Lastly, mode estimation and segmentation are performed by adaptive mean shift and separated clustering method respectively. This approach is automatically done, thereby allowing to easily fill-in corrupted regions containing completely different structures and surrounding backgrounds. Applications of proposed method include the restoration of damaged text images; removal of superimposed noises or streaks. We so can see that proposed approach is efficient and robust in terms of restoring and segmenting text images corrupted.

Big Data Analysis of News on Purchasing Second-hand Clothing and Second-hand Luxury Goods: Identification of Social Perception and Current Situation Using Text Mining (중고의류와 중고명품 구매 관련 언론 보도 빅데이터 분석: 텍스트마이닝을 활용한 사회적 인식과 현황 파악)

  • Hwa-Sook Yoo
    • Human Ecology Research
    • /
    • v.61 no.4
    • /
    • pp.687-707
    • /
    • 2023
  • This study was conducted to obtain useful information on the development of the future second-hand fashion market by obtaining information on the current situation through unstructured text data distributed as news articles related to 'purchase of second-hand clothing' and 'purchase of second-hand luxury goods'. Text-based unstructured data was collected on a daily basis from Naver news from January 1st to December 31st, 2022, using 'purchase of second-hand clothing' and 'purchase of second-hand luxury goods' as collection keywords. This was analyzed using text mining, and the results are as follows. First, looking at the frequency, the collection data related to the purchase of second-hand luxury goods almost quadrupled compared to the data related to the purchase of second-hand clothing, indicating that the purchase of second-hand luxury goods is receiving more social attention. Second, there were common words between the data obtained by the two collection keywords, but they had different words. Regarding second-hand clothing, words related to donations, sharing, and compensation sales were mainly mentioned, indicating that the purchase of second-hand clothing tends to be recognized as an eco-friendly transaction. In second-hand luxury goods, resale and genuine controversy related to the transaction of second-hand luxury goods, second-hand trading platforms, and luxury brands were frequently mentioned. Third, as a result of clustering, data related to the purchase of second-hand clothing were divided into five groups, and data related to the purchase of second-hand luxury goods were divided into six groups.

Topic Analysis of the National Petition Site and Prediction of Answerable Petitions Based on Deep Learning (국민청원 주제 분석 및 딥러닝 기반 답변 가능 청원 예측)

  • Woo, Yun Hui;Kim, Hyon Hee
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.9 no.2
    • /
    • pp.45-52
    • /
    • 2020
  • Since the opening of the national petition site, it has attracted much attention. In this paper, we perform topic analysis of the national petition site and propose a prediction model for answerable petitions based on deep learning. First, 1,500 petitions are collected, topics are extracted based on the petitions' contents. Main subjects are defined using K-means clustering algorithm, and detailed subjects are defined using topic modeling of petitions belonging to the main subjects. Also, long short-term memory (LSTM) is used for prediction of answerable petitions. Not only title and contents but also categories, length of text, and ratio of part of speech such as noun, adjective, adverb, verb are also used for the proposed model. Our experimental results show that the type 2 model using other features such as ratio of part of speech, length of text, and categories outperforms the type 1 model without other features.