• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.26 no.7
• /
• pp.949-955
• /
• 2022

Deep learning-based text summarization models are not free from datasets. For example, a summarization model trained with a news summarization dataset is not good at summarizing other types of texts such as internet posts and papers. In this study, we define this phenomenon as Data Bias Problem (DBP) and propose two training methods for solving it. The first is the 'proper nouns masking' that masks proper nouns. The second is the 'length variation' that randomly inflates or deflates the length of text. As a result, experiments show that our methods are efficient for solving DBP. In addition, we analyze the results of the experiments and present future development directions. Our contributions are as follows: (1) We discovered DBP and defined it for the first time. (2) We proposed two efficient training methods and conducted actual experiments. (3) Our methods can be applied to all summarization models and are easy to implement, so highly practical.

• The Journal of the Convergence on Culture Technology
• /
• v.7 no.4
• /
• pp.745-750
• /
• 2021

In Big data visualization analysis of unstructured text data, raw data is mostly large-capacity, and analysis techniques cannot be applied without cleansing it unstructured. Therefore, from the collected raw data, unnecessary data is removed through the first heuristic cleansing process and Stopwords are removed through the second machine cleansing process. Then, the frequency of the vocabulary is calculated, visualized using the word cloud technique, and key issues are extracted and informationalized, and the results are analyzed. In this study, we propose a new Stopword cleansing technique using an external Stopword set (DB) in Python word cloud, and derive the problems and effectiveness of this technique through practical case analysis. And, through this verification result, the utility of the practical application of word cloud analysis applying the proposed cleansing technique is presented.

• The Journal of The Korea Institute of Intelligent Transport Systems
• /
• v.22 no.2
• /
• pp.182-194
• /
• 2023

Traffic impact assessment plays a crucial role in resolving traffic issues that may arise during the implementation of urban and transportation projects. However, reported results diverge, presumably because the items reviewed differ. In this study, we analyze traffic improvement measures approved for traffic impact assessment, identify key items, and present items that should be included in assessments. Specifically, TF-IDF and N-gram analysis and text mining were performed with focus on urban development projects approved in Gyeonggi Province. The results obtained show that keywords associated with newly established transportation infrastructure, such as roads and intersections, were essential assessment items, followed by the locations of entrances and exits and pedestrian connectivity. We recommend that considerations of the items presented in this study be incorporated into future traffic impact assessment guidelines and standards to improve the consistency and objectivity of the assessment process.

• 한국정보교육학회:학술대회논문집
• /
• 2021.08a
• /
• pp.327-333
• /
• 2021

Currently development of the Artificial Intelligence technology is rapidly progressing in the era of the Fourth Industrial Revolution. The government is trying to improve the education of Artificial Intelligence and cultivating human resources. However there are very few cases where A.I technology is actually used in public education classes. Therefore we designed a text repository by implementing A.I speech recognition to provide career counseling for elementary school students. In the meantime, there have been many difficulties in giving advance consultations required for students' career counseling. In this study we suggested A.I speech recognition technology which can solve addressed problem and we planned various ways to make the program more educational. To conclude we expect A.I technology implemented in this study provides effective solution to career counseling.

• 한국정보교육학회:학술대회논문집
• /
• 2021.08a
• /
• pp.141-148
• /
• 2021

The mainstream tool for software education for elementary students is Educational Programming Language. It is essential for upper graders to advance from EPL to text based programming language. However, many students experience difficulty in adopting to this change since Python is run in English. Python is an actively used TPL. This study focuses on developing an education program to facilitate learning Python for Korean speaking students. We have extracted the necessary reserved words needed for data analysis in Python. Then we replaced the extracted words into Korean terms that could be understood in elementary level. The replaced terms were matched on one-to-one correspondence with reserved words used in Python. This devised program would assist students in experiencing data analysis with Python. We expect that this education program will be applied effectively as a basic resource to learn TPL.

• Korean Journal of Cognitive Science
• /
• v.34 no.3
• /
• pp.197-226
• /
• 2023

Generative Text Summarization is one of the Natural Language Processing tasks. It generates a short abbreviated summary while preserving the content of the long text. ROUGE is a widely used lexical-overlap based metric for text summarization models in generative summarization benchmarks. Although it shows very high performance, the studies report that 30% of the generated summary and the text are still inconsistent. This paper proposes a methodology for evaluating the performance of the summary model without using the correct summary. AggreFACT is a human-annotated dataset that classifies the types of errors in neural text summarization models. Among all the test candidates, the two cases, generation summary, and when errors occurred throughout the summary showed the highest correlation results. We observed that the proposed evaluation score showed a high correlation with models finetuned with BART and PEGASUS, which is pretrained with a large-scale Transformer structure.

• Journal of Korea Society of Industrial Information Systems
• /
• v.28 no.6
• /
• pp.63-81
• /
• 2023

This study endeavors to classify and categorize similar policy programs through network clustering analysis, using textual information from data-related policy programs in Korea. To achieve this, descriptions of data-related budgetary programs in South Korea in 2022 were collected, and keywords from the program contents were extracted. Subsequently, the similarity between each program was derived using TF-IDF, and policy program network was constructed accordingly. Following this, the structural characteristics of the network were analyzed, and similar policy programs were clustered and categorized through network clustering. Upon analyzing a total of 97 programs, 7 major clusters were identified, signifying that programs with analogous themes or objectives were categorized based on application area or services utilizing data. The findings of this research illuminate the current status of data-related policy programs in Korea, providing policy implications for a strategic approach to planning future national data strategies and programs, and contributing to the establishment of evidence-based policies.

• The Journal of the Convergence on Culture Technology
• /
• v.9 no.6
• /
• pp.935-940
• /
• 2023

Most text data collected through web scraping for artificial intelligence and big data analysis is generally large and unstructured, so a purification process is required for big data analysis. The process becomes structured data that can be analyzed through a heuristic pre-processing refining step and a post-processing machine refining step. Therefore, in this study, in the post-processing machine refining process, the Korean dictionary and the stopword dictionary are used to extract vocabularies for frequency analysis for word cloud analysis. In this process, "user-defined stopwords" are used to efficiently remove stopwords that were not removed. We propose a methodology for applying the "thesaurus" and examine the pros and cons of the proposed refining method through a case analysis using the "user-defined stop word thesaurus" technique proposed to complement the problems of the existing "stop word dictionary" method with R's word cloud technique. We present comparative verification and suggest the effectiveness of practical application of the proposed methodology.

• Journal of the Korean Applied Science and Technology
• /
• v.40 no.4
• /
• pp.724-732
• /
• 2023

In this study, we tried to analyze the characteristics of color cosmetics information search and the major information of interest in the color cosmetics market after COVID-19 shown in the text mining analysis results by collecting data on online interest information of consumers in the color cosmetics market after COVID-19. In the empirical analysis, text mining was performed on all documents such as news, blogs, cafes, and web pages, including the word "color cosmetics". As a result of the analysis, online information searches for color cosmetics after COVID-19 were mainly focused on purchase information, information on skin and mask-related makeup methods, and major topics such as interest brands and event information. As a result, post-COVID-19 color cosmetics buyers will become more sensitive to purchase information such as product value, safety, price benefits, and store information through active online information search, so a response strategy is required.

• The Journal of the Convergence on Culture Technology
• /
• v.10 no.1
• /
• pp.609-615
• /
• 2024

The construction industry's fatality count stands at 402, comprising approximately 46% of total industrial accidents. Notably, construction costs less than 5 billion won account for about 69%, so strengthening safety management at small and medium-sized construction sites is required. In this study, 19,511 accident investigation data were collected using web scraping. Through statistical analysis of the collected structured data and text mining analysis of the unstructured data, accident types and causes of accidents were analyzed by construction costs at sites less than 5 billion won. As a result, it was confirmed that there were differences in accident types and causes depending on the construction costs. It is hoped that the results of this study will be used for customized safety management at small and medium-sized construction sites.