• Title/Summary/Keyword: Data log

Search Result 2,118, Processing Time 0.028 seconds

THE STUDY OF FLOOD FREQUENCY ESTIMATES USING CAUCHY VARIABLE KERNEL

  • Moon, Young-Il;Cha, Young-Il;Ashish Sharma
    • Water Engineering Research
    • /
    • v.2 no.1
    • /
    • pp.1-10
    • /
    • 2001
  • The frequency analyses for the precipitation data in Korea were performed. We used daily maximum series, monthly maximum series, and annual series. For nonparametric frequency analyses, variable kernel estimators were used. Nonparametric methods do not require assumptions about the underlying populations from which the data are obtained. Therefore, they are better suited for multimodal distributions with the advantage of not requiring a distributional assumption. In order to compare their performance with parametric distributions, we considered several probability density functions. They are Gamma, Gumbel, Log-normal, Log-Pearson type III, Exponential, Generalized logistic, Generalized Pareto, and Wakeby distributions. The variable kernel estimates are comparable and are in the middle of the range of the parametric estimates. The variable kernel estimates show a very small probability in extrapolation beyond the largest observed data in the sample. However, the log-variable kernel estimates remedied these defects with the log-transformed data.

  • PDF

Prediction of Cognitive Ability Utilizing a Machine Learning approach based on Digital Therapeutics Log Data

  • Yeojin Kim;Jiseon Yang;Dohyoung Rim;Uran Oh
    • International journal of advanced smart convergence
    • /
    • v.12 no.2
    • /
    • pp.17-24
    • /
    • 2023
  • Given the surge in the elderly population, and increasing in dementia cases, there is a growing interest in digital therapies that facilitate steady remote treatment. However, in the cognitive assessment of digital therapies through clinical trials, the absence of log data as an essential evaluation factor is a significant issue. To address this, we propose a solution of utilizing weighted derived variables based on high-importance variables' accuracy in log data utilization as an indirect cognitive assessment factor for digital therapies. We have validated the effectiveness of this approach using machine learning techniques such as XGBoost, LGBM, and CatBoost. Thus, we suggest the use of log data as a rapid and indirect cognitive evaluation factor for digital therapy users.

Spark-based Network Log Analysis Aystem for Detecting Network Attack Pattern Using Snort (Snort를 이용한 비정형 네트워크 공격패턴 탐지를 수행하는 Spark 기반 네트워크 로그 분석 시스템)

  • Baek, Na-Eun;Shin, Jae-Hwan;Chang, Jin-Su;Chang, Jae-Woo
    • The Journal of the Korea Contents Association
    • /
    • v.18 no.4
    • /
    • pp.48-59
    • /
    • 2018
  • Recently, network technology has been used in various fields due to development of network technology. However, there has been an increase in the number of attacks targeting public institutions and companies by exploiting the evolving network technology. Meanwhile, the existing network intrusion detection system takes much time to process logs as the amount of network log increases. Therefore, in this paper, we propose a Spark-based network log analysis system that detects unstructured network attack pattern. by using Snort. The proposed system extracts and analyzes the elements required for network attack pattern detection from large amount of network log data. For the analysis, we propose a rule to detect network attack patterns for Port Scanning, Host Scanning, DDoS, and worm activity, and can detect real attack pattern well by applying it to real log data. Finally, we show from our performance evaluation that the proposed Spark-based log analysis system is more than two times better on log data processing performance than the Hadoop-based system.

A Framework for Web Log Analysis Using Process Mining Techniques (프로세스 마이닝을 이용한 웹 로그 분석 프레임워크)

  • Ahn, Yunha;Oh, Kyuhyup;Kim, Sang-Kuk;Jung, Jae-Yoon
    • Journal of Information Technology and Architecture
    • /
    • v.11 no.1
    • /
    • pp.25-32
    • /
    • 2014
  • Web mining techniques are often used to discover useful patterns from data log generated by Web servers for the purpose of web usage analysis. Yet traditional Web mining techniques do not reflect sufficiently sequential properties of Web log data. To address such weakness, we introduce a framework for analyzing Web access log data by using process mining techniques. To illustrate the proposed framework, we show the analysis of Web access log in a campus information system based on the framework and discuss the implication of the analysis result.

Design of Log Management System based on Document Database for Big Data Management (빅데이터 관리를 위한 문서형 DB 기반 로그관리 시스템 설계)

  • Ryu, Chang-ju;Han, Myeong-ho;Han, Seung-jo
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.19 no.11
    • /
    • pp.2629-2636
    • /
    • 2015
  • Recently Big Data management have a rapid increases interest in IT field, much research conducting to solve a problem of real-time processing to Big Data. Lots of resources are required for the ability to store data in real-time over the network but there is the problem of introducing an analyzing system due to aspect of high cost. Need of redesign of the system for low cost and high efficiency had been increasing to solve the problem. In this paper, the document type of database, MongoDB, is used for design a log management system based a document type of database, that is good at big data managing. The suggested log management system is more efficient than other method on log collection and processing, and it is strong on data forgery through the performance evaluation.

Outlying Cell Identification Method Using Interaction Estimates of Log-linear Models

  • Hong, Chong Sun;Jung, Min Jung
    • Communications for Statistical Applications and Methods
    • /
    • v.10 no.2
    • /
    • pp.291-303
    • /
    • 2003
  • This work is proposed an alternative identification method of outlying cell which is one of important issues in categorical data analysis. One finds that there is a strong relationship between the location of an outlying cell and the corresponding parameter estimates of the well-fitted log-linear model. Among parameters of log-linear model, an outlying cell is affected by interaction terms rather than main effect terms. Hence one could identify an outlying cell by investigating of parameter estimates in an appropriate log-linear model.

A study on log-density with log-odds graph for variable selection in logistic regression (로지스틱회귀모형의 변수선택에서 로그-오즈 그래프를 통한 로그-밀도비 연구)

  • Kahng, Myung-Wook;Shin, Eun-Young
    • Journal of the Korean Data and Information Science Society
    • /
    • v.23 no.1
    • /
    • pp.99-111
    • /
    • 2012
  • The log-density ratio of the conditional densities of the predictors given the response variable provides useful information for variable selection in the logistic regression model. In this paper, we consider the predictors that are needed and how they should be included in the model. If the conditional distributions are skewed, the distributions can be considered as gamma distributions. Under this assumption, linear and log terms are generally included in the model. The log-odds graph is a very useful graphical tool in this study. A graphical study is presented which shows that if the conditional distributions of x|y for the two groups overlap significantly, we need both the linear and quadratic terms. On the contrary, if they are well separated, only the linear or log term is needed in the model.

EDF: An Interactive Tool for Event Log Generation for Enabling Process Mining in Small and Medium-sized Enterprises

  • Frans Prathama;Seokrae Won;Iq Reviessay Pulshashi;Riska Asriana Sutrisnowati
    • Journal of the Korea Society of Computer and Information
    • /
    • v.29 no.6
    • /
    • pp.101-112
    • /
    • 2024
  • In this paper, we present EDF (Event Data Factory), an interactive tool designed to assist event log generation for process mining. EDF integrates various data connectors to improve its capability to assist users in connecting to diverse data sources. Our tool employs low-code/no-code technology, along with graph-based visualization, to help non-expert users understand process flow and enhance the user experience. By utilizing metadata information, EDF allows users to efficiently generate an event log containing case, activity, and timestamp attributes. Through log quality metrics, our tool enables users to assess the generated event log quality. We implement EDF under a cloud-based architecture and run a performance evaluation. Our case study and results demonstrate the usability and applicability of EDF. Finally, an observational study confirms that EDF is easy to use and beneficial, expanding small and medium-sized enterprises' (SMEs) access to process mining applications.

A Study on the Intrusion Detection Method using Firewall Log (방화벽 로그를 이용한 침입탐지기법 연구)

  • Yoon, Sung-Jong;Kim, Jeong-Ho
    • Journal of Information Technology Applications and Management
    • /
    • v.13 no.4
    • /
    • pp.141-153
    • /
    • 2006
  • According to supply of super high way internet service, importance of security becomes more emphasizing. Therefore, flawless security solution is needed for blocking information outflow when we send or receive data. large enterprise and public organizations can react to this problem, however, small organization with limited work force and capital can't. Therefore they need to elevate their level of information security by improving their information security system without additional money. No hackings can be done without passing invasion blocking system which installed at the very front of network. Therefore, if we manage.isolation log effective, we can recognize hacking trial at the step of pre-detection. In this paper, it supports information security manager to execute isolation log analysis very effectively. It also provides isolation log analysis module which notifies hacking attack by analyzing isolation log.

  • PDF

Anomaly Detection Technique of Log Data Using Hadoop Ecosystem (하둡 에코시스템을 활용한 로그 데이터의 이상 탐지 기법)

  • Son, Siwoon;Gil, Myeong-Seon;Moon, Yang-Sae
    • KIISE Transactions on Computing Practices
    • /
    • v.23 no.2
    • /
    • pp.128-133
    • /
    • 2017
  • In recent years, the number of systems for the analysis of large volumes of data is increasing. Hadoop, a representative big data system, stores and processes the large data in the distributed environment of multiple servers, where system-resource management is very important. The authors attempted to detect anomalies from the rapid changing of the log data that are collected from the multiple servers using simple but efficient anomaly-detection techniques. Accordingly, an Apache Hive storage architecture was designed to store the log data that were collected from the multiple servers in the Hadoop ecosystem. Also, three anomaly-detection techniques were designed based on the moving-average and 3-sigma concepts. It was finally confirmed that all three of the techniques detected the abnormal intervals correctly, while the weighted anomaly-detection technique is more precise than the basic techniques. These results show an excellent approach for the detection of log-data anomalies with the use of simple techniques in the Hadoop ecosystem.