• Title/Summary/Keyword: Open data mining

Search Result 116, Processing Time 0.022 seconds

Generating and Controlling an Interlinking Network of Technical Terms to Enhance Data Utilization (데이터 활용률 제고를 위한 기술 용어의 상호 네트워크 생성과 통제)

  • Jeong, Do-Heon
    • Journal of the Korean Society for information Management
    • /
    • v.35 no.1
    • /
    • pp.157-182
    • /
    • 2018
  • As data management and processing techniques have been developed rapidly in the era of big data, nowadays a lot of business companies and researchers have been interested in long tail data which were ignored in the past. This study proposes methods for generating and controlling a network of technical terms based on text mining technique to enhance data utilization in the distribution of long tail theory. Especially, an edit distance technique of text mining has given us efficient methods to automatically create an interlinking network of technical terms in the scholarly field. We have also used linked open data system to gather experimental data to improve data utilization and proposed effective methods to use data of LOD systems and algorithm to recognize patterns of terms. Finally, the performance evaluation test of the network of technical terms has shown that the proposed methods were useful to enhance the rate of data utilization.

A study on unstructured text mining algorithm through R programming based on data dictionary (Data Dictionary 기반의 R Programming을 통한 비정형 Text Mining Algorithm 연구)

  • Lee, Jong Hwa;Lee, Hyun-Kyu
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.20 no.2
    • /
    • pp.113-124
    • /
    • 2015
  • Unlike structured data which are gathered and saved in a predefined structure, unstructured text data which are mostly written in natural language have larger applications recently due to the emergence of web 2.0. Text mining is one of the most important big data analysis techniques that extracts meaningful information in the text because it has not only increased in the amount of text data but also human being's emotion is expressed directly. In this study, we used R program, an open source software for statistical analysis, and studied algorithm implementation to conduct analyses (such as Frequency Analysis, Cluster Analysis, Word Cloud, Social Network Analysis). Especially, to focus on our research scope, we used keyword extract method based on a Data Dictionary. By applying in real cases, we could find that R is very useful as a statistical analysis software working on variety of OS and with other languages interface.

Comparison of Hoek-Brown and Mohr-Coulomb failure criterion for deep open coal mine slope stability

  • Aksoy, Cemalettin O.;Uyar, Guzin G.;Ozcelik, Yilmaz
    • Structural Engineering and Mechanics
    • /
    • v.60 no.5
    • /
    • pp.809-828
    • /
    • 2016
  • In deep open pit mines, slope stability is very important. Particularly, increasing the depths increase the risks in mines having weak rock mass. Blasting operations in this type of open pits may have a negative impact on slope stability. Several or combination of methods can be used in order to enable better analysis in this type of deep open-pit mines. Numerical modeling is one of these options. Many complex problems can be integrated into numerical methods at the same time and analysis, solutions can be performed on a single model. Rock failure criterions and rock models are used in numerical modeling. Hoek-Brown and Mohr-Coulomb terms are the two most commonly used rock failure conditions. In this study, mine planning and discontinuity conditions of a lignite mine facing two big landslides previously, has been investigated. Moreover, the presence of some damage before starting the study was identified in surrounding structures. The primary research of this study is on slope study. In slope stability analysis, numerical modeling methods with Hoek-Brown and Mohr-Coulomb failure criterions were used separately. Preparing the input data to the numerical model, the outcomes of patented-blast vibration minimization method, developed by co-author was used. The analysis showed that, the model prepared by applying Hoek-Brown failure criterion, failed in the stage of 10. However, the model prepared by using Mohr-Coulomb failure criterion did not fail even in the stage 17. Examining the full research field, there has been ongoing production in this mine without any failure and damage to surface structures.

Design of Efficient Query Language to support Local information administration environment (지역정보 관리 환경을 지원하기 위한 효율적인 질의 언어의 설계)

  • Kang, Sung-Kwan;Rhee, Phill-Kyu
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2008.06c
    • /
    • pp.36-40
    • /
    • 2008
  • SIMS manages data for various spatial and non-spatial as integral management system to support space information administration environment and support several application works. Without being limited to spatial data that existent spatial Data Mining question language advances handling in this paper, did so that can find useful information from various data connected with automatically data collection, artificial satellite side upside service, remote sensing, GPS. Mobile Computing and data about Spatio-Temporal. Also, we designed spatial Data Mining query language that support a spatial Data Mining exclusive use system based on SIMS.

  • PDF

A Comparative Study on the Performance of Intrusion Detection using Decision Tree and Artificial Neural Network Models (의사결정트리와 인공 신경망 기법을 이용한 침입탐지 효율성 비교 연구)

  • Jo, Seongrae;Sung, Haengnam;Ahn, Byunghyuk
    • Journal of Korea Society of Digital Industry and Information Management
    • /
    • v.11 no.4
    • /
    • pp.33-45
    • /
    • 2015
  • Currently, Internet is used an essential tool in the business area. Despite this importance, there is a risk of network attacks attempting collection of fraudulence, private information, and cyber terrorism. Firewalls and IDS(Intrusion Detection System) are tools against those attacks. IDS is used to determine whether a network data is a network attack. IDS analyzes the network data using various techniques including expert system, data mining, and state transition analysis. This paper tries to compare the performance of two data mining models in detecting network attacks. They are decision tree (C4.5), and neural network (FANN model). I trained and tested these models with data and measured the effectiveness in terms of detection accuracy, detection rate, and false alarm rate. This paper tries to find out which model is effective in intrusion detection. In the analysis, I used KDD Cup 99 data which is a benchmark data in intrusion detection research. I used an open source Weka software for C4.5 model, and C++ code available for FANN model.

OryzaGP: rice gene and protein dataset for named-entity recognition

  • Larmande, Pierre;Do, Huy;Wang, Yue
    • Genomics & Informatics
    • /
    • v.17 no.2
    • /
    • pp.17.1-17.3
    • /
    • 2019
  • Text mining has become an important research method in biology, with its original purpose to extract biological entities, such as genes, proteins and phenotypic traits, to extend knowledge from scientific papers. However, few thorough studies on text mining and application development, for plant molecular biology data, have been performed, especially for rice, resulting in a lack of datasets available to solve named-entity recognition tasks for this species. Since there are rare benchmarks available for rice, we faced various difficulties in exploiting advanced machine learning methods for accurate analysis of the rice literature. To evaluate several approaches to automatically extract information from gene/protein entities, we built a new dataset for rice as a benchmark. This dataset is composed of a set of titles and abstracts, extracted from scientific papers focusing on the rice species, and is downloaded from PubMed. During the 5th Biomedical Linked Annotation Hackathon, a portion of the dataset was uploaded to PubAnnotation for sharing. Our ultimate goal is to offer a shared task of rice gene/protein name recognition through the BioNLP Open Shared Tasks framework using the dataset, to facilitate an open comparison and evaluation of different approaches to the task.

The impact of the change in the splitting method of decision trees on the prediction power (의사결정나무의 분기법 변화가 예측력에 미치는 영향)

  • Chang, Youngjae
    • The Korean Journal of Applied Statistics
    • /
    • v.35 no.4
    • /
    • pp.517-525
    • /
    • 2022
  • In the era of big data, various data mining techniques have been proposed as major analysis methodologies. As complex and diverse data is mass-produced, data mining techniques have attracted attention as a method that forms the foundation of data science. In this paper, we focused on the decision tree, which is frequently used in practice and easy to understand as one of representative data mining methods. Specifically, we analyzed the effect of the splitting method of decision trees on the model performance. We compared the prediction power and structures of decision tree models with different split methods based on various simulated data. The results show that the linear combination split method can improve the prediction accuracy of decision trees in the case of data simulated from nonlinear models with complex structure.

Detection of Malicious Code using Association Rule Mining and Naive Bayes classification (연관규칙 마이닝과 나이브베이즈 분류를 이용한 악성코드 탐지)

  • Ju, Yeongji;Kim, Byeongsik;Shin, Juhyun
    • Journal of Korea Multimedia Society
    • /
    • v.20 no.11
    • /
    • pp.1759-1767
    • /
    • 2017
  • Although Open API has been invigorated by advancements in the software industry, diverse types of malicious code have also increased. Thus, many studies have been carried out to discriminate the behaviors of malicious code based on API data, and to determine whether malicious code is included in a specific executable file. Existing methods detect malicious code by analyzing signature data, which requires a long time to detect mutated malicious code and has a high false detection rate. Accordingly, in this paper, we propose a method that analyzes and detects malicious code using association rule mining and an Naive Bayes classification. The proposed method reduces the false detection rate by mining the rules of malicious and normal code APIs in the PE file and grouping patterns using the DHP(Direct Hashing and Pruning) algorithm, and classifies malicious and normal files using the Naive Bayes.

An Exploratory Analysis of Online Discussion of Library and Information Science Professionals in India using Text Mining

  • Garg, Mohit;Kanjilal, Uma
    • Journal of Information Science Theory and Practice
    • /
    • v.10 no.3
    • /
    • pp.40-56
    • /
    • 2022
  • This paper aims to implement a topic modeling technique for extracting the topics of online discussions among library professionals in India. Topic modeling is the established text mining technique popularly used for modeling text data from Twitter, Facebook, Yelp, and other social media platforms. The present study modeled the online discussions of Library and Information Science (LIS) professionals posted on Lis Links. The text data of these posts was extracted using a program written in R using the package "rvest." The data was pre-processed to remove blank posts, posts having text in non-English fonts, punctuation, URLs, emails, etc. Topic modeling with the Latent Dirichlet Allocation algorithm was applied to the pre-processed corpus to identify each topic associated with the posts. The frequency analysis of the occurrence of words in the text corpus was calculated. The results found that the most frequent words included: library, information, university, librarian, book, professional, science, research, paper, question, answer, and management. This shows that the LIS professionals actively discussed exams, research, and library operations on the forum of Lis Links. The study categorized the online discussions on Lis Links into ten topics, i.e. "LIS Recruitment," "LIS Issues," "Other Discussion," "LIS Education," "LIS Research," "LIS Exams," "General Information related to Library," "LIS Admission," "Library and Professional Activities," and "Information Communication Technology (ICT)." It was found that the majority of the posts belonged to "LIS Exam," followed by "Other Discussions" and "General Information related to the Library."

Implementation of Association Rules Creation System from GML Documents (GML 문서에서 연관규칙 생성 시스템 구현)

  • Kim, Eui-Chan;Hwang, Byung-Yeon
    • Journal of Korea Spatial Information System Society
    • /
    • v.8 no.1 s.16
    • /
    • pp.27-35
    • /
    • 2006
  • As the increasing interest about geographical information, such researches and applied fields become wide. OGC(Open GIS Consortium) developed GML(Geography Markup Language) which is adopted XML(extensible Markup Language) in GIS field. In various applied field, GML is used and studied continuously. This paper try to find out the meaningful rules using Apriori algorithm from GML documents, one of the data mining techniques which is studied based on existing XML documents There are two ways to find out the rules. One is the way that find out the related rules as extracting the content in GML documents, the other find out the related rules based on used tags and attributes. This paper describes searching the rules through two ways and shows the system adopted two ways.

  • PDF