• Title/Summary/Keyword: Log preprocessing

Search Result 27, Processing Time 0.032 seconds

The Analysis of Individual Learning Status on Web-Based Instruction (웹기반 교육에서 학습자별 학습현황 분석에 관한 연구)

  • Shin, Ji-Yeun;Jeong, Ok-Ran;Cho, Dong-Sub
    • The Journal of Korean Association of Computer Education
    • /
    • v.6 no.2
    • /
    • pp.107-120
    • /
    • 2003
  • In Web Based Instruction, as evaluation of learning process means individual student's learning activity, it demands data on learning time, pattern, participation, environment in a specific learning contents. The purpose of this paper is to reflect analysis results of individual student's learning status in achievement evaluation using the most suitable web log mining to settle evaluation problem of learning process, an issue in web based instruction. The contents and results of this study are as following. First, conformity item for learning status analysis is determined and web log data preprocessing is executed. Second, on the basis of web log data, I construct student's database and analyze learning status using data mining techniques.

  • PDF

Fuzzy Web Usage Mining for User Modeling

  • Jang, Jae-Sung;Jun, Sung-Hae;Oh, Kyung-Whan
    • International Journal of Fuzzy Logic and Intelligent Systems
    • /
    • v.2 no.3
    • /
    • pp.204-209
    • /
    • 2002
  • The interest of data mining in artificial intelligence with fuzzy logic has been increased. Data mining is a process of extracting desirable knowledge and interesting pattern ken large data set. Because of expansion of WWW, web data is more and more huge. Besides mining web contents and web structures, another important task for web mining is web usage mining which mines web log data to discover user access pattern. The goal of web usage mining in this paper is to find interesting user pattern in the web with user feedback. It is very important to find user's characteristic fer e-business environment. In Customer Relationship Management, recommending product and sending e-mail to user by extracted users characteristics are needed. Using our method, we extract user profile from the result of web usage mining. In this research, we concentrate on finding association rules and verify validity of them. The proposed procedure can integrate fuzzy set concept and association rule. Fuzzy association rule uses given server log file and performs several preprocessing tasks. Extracted transaction files are used to find rules by fuzzy web usage mining. To verify the validity of user's feedback, the web log data from our laboratory web server.

Impedance Estimation from 3-D Seismic Data (3차원 탄성파로부터 매질의 임피던스 산출에 관한 연구)

  • Lee, Doo-Sung
    • Geophysics and Geophysical Exploration
    • /
    • v.3 no.1
    • /
    • pp.7-12
    • /
    • 2000
  • The paper discusses a data processing methodology that derives a three dimensional porosity volume information from the 3-D seismic dataset. The methodology consists of preprocessing and inversion procedures. The purpose of the preprocessing is balancing the amplitudes of seismic traces by using reflectivity series derived from sonic and density logs. There are eight sonic logs are available in the study area; therefore, we can compute only 8 balance functions. The balance function for every seismic trace was derived from these 8 balance functions by kriging. In order to derive a wide-band acoustic impedance --similar to the one can be derived from a sonic log- from a band-limited reflection seismogram, we need to recover missing low- and high-frequency information of the seismic trace. For that Purpose we use the autoregressive method.

  • PDF

A Convex Layer Tree for the Ray-Shooting Problem (광선 슈팅 문제를 위한 볼록 레이어 트리)

  • Kim, Soo-Hwan
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.21 no.4
    • /
    • pp.753-758
    • /
    • 2017
  • The ray-shooting problem is to find the first intersection point on the surface of given geometric objects where a ray moving along a straight line hits. Since rays are usually given in the form of queries, this problem is typically solved as follows. First, a data structure for a collection of objects is constructed as preprocessing. Then, the answer for each query ray is quickly computed using the data structure. In this paper, we consider the ray-shooting problem about the set of vertical line segments on the x-axis. We present a new data structure called a convex layer tree for n vertical line segments given by input. This is a tree structure consisting of layers of convex hulls of vertical line segments. It can be constructed in O(n log n) time and O(n) space and is easy to implement. We also present an algorithm to solve each query in O(log n) time using this data structure.

A MapReduce-Based Workflow BIG-Log Clustering Technique (맵리듀스기반 워크플로우 빅-로그 클러스터링 기법)

  • Jin, Min-Hyuck;Kim, Kwanghoon Pio
    • Journal of Internet Computing and Services
    • /
    • v.20 no.1
    • /
    • pp.87-96
    • /
    • 2019
  • In this paper, we propose a MapReduce-supported clustering technique for collecting and classifying distributed workflow enactment event logs as a preprocessing tool. Especially, we would call the distributed workflow enactment event logs as Workflow BIG-Logs, because they are satisfied with as well as well-fitted to the 5V properties of BIG-Data like Volume, Velocity, Variety, Veracity and Value. The clustering technique we develop in this paper is intentionally devised for the preprocessing phase of a specific workflow process mining and analysis algorithm based upon the workflow BIG-Logs. In other words, It uses the Map-Reduce framework as a Workflow BIG-Logs processing platform, it supports the IEEE XES standard data format, and it is eventually dedicated for the preprocessing phase of the ${\rho}$-Algorithm that is a typical workflow process mining algorithm based on the structured information control nets. More precisely, The Workflow BIG-Logs can be classified into two types: of activity-based clustering patterns and performer-based clustering patterns, and we try to implement an activity-based clustering pattern algorithm based upon the Map-Reduce framework. Finally, we try to verify the proposed clustering technique by carrying out an experimental study on the workflow enactment event log dataset released by the BPI Challenges.

A Text Mining-based Intrusion Log Recommendation in Digital Forensics (디지털 포렌식에서 텍스트 마이닝 기반 침입 흔적 로그 추천)

  • Ko, Sujeong
    • KIPS Transactions on Computer and Communication Systems
    • /
    • v.2 no.6
    • /
    • pp.279-290
    • /
    • 2013
  • In digital forensics log files have been stored as a form of large data for the purpose of tracing users' past behaviors. It is difficult for investigators to manually analysis the large log data without clues. In this paper, we propose a text mining technique for extracting intrusion logs from a large log set to recommend reliable evidences to investigators. In the training stage, the proposed method extracts intrusion association words from a training log set by using Apriori algorithm after preprocessing and the probability of intrusion for association words are computed by combining support and confidence. Robinson's method of computing confidences for filtering spam mails is applied to extracting intrusion logs in the proposed method. As the results, the association word knowledge base is constructed by including the weights of the probability of intrusion for association words to improve the accuracy. In the test stage, the probability of intrusion logs and the probability of normal logs in a test log set are computed by Fisher's inverse chi-square classification algorithm based on the association word knowledge base respectively and intrusion logs are extracted from combining the results. Then, the intrusion logs are recommended to investigators. The proposed method uses a training method of clearly analyzing the meaning of data from an unstructured large log data. As the results, it complements the problem of reduction in accuracy caused by data ambiguity. In addition, the proposed method recommends intrusion logs by using Fisher's inverse chi-square classification algorithm. So, it reduces the rate of false positive(FP) and decreases in laborious effort to extract evidences manually.

Fast Thinning Algorithm based on Improved SOG($SOG^*$) (개선된 SOG 기반 고속 세선화 알고리즘($SOG^*$))

  • Lee, Chan-Hui;Jeong, Sun-Ho
    • The KIPS Transactions:PartB
    • /
    • v.8B no.6
    • /
    • pp.651-656
    • /
    • 2001
  • In this paper, we propose Improved Self-Organized Graph(Improved SOG:$SOG^*$)thinning method, which maintains the excellent thinning results of Self-organized graph(SOG) built from Self-Organizing features map and improves the performance of modified SOG using a new incremental learning method of Kohonen features map. In the experiments, this method shows the thinning results equal to those of SOG and the time complexity O((logM)3) superior to it. Therefore, the proposed method is useful for the feature extraction from digits and characters in the preprocessing step.

  • PDF

Resource Metric Refining Module for AIOps Learning Data in Kubernetes Microservice

  • Jonghwan Park;Jaegi Son;Dongmin Kim
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.17 no.6
    • /
    • pp.1545-1559
    • /
    • 2023
  • In the cloud environment, microservices are implemented through Kubernetes, and these services can be expanded or reduced through the autoscaling function under Kubernetes, depending on the service request or resource usage. However, the increase in the number of nodes or distributed microservices in Kubernetes and the unpredictable autoscaling function make it very difficult for system administrators to conduct operations. Artificial Intelligence for IT Operations (AIOps) supports resource management for cloud services through AI and has attracted attention as a solution to these problems. For example, after the AI model learns the metric or log data collected in the microservice units, failures can be inferred by predicting the resources in future data. However, it is difficult to construct data sets for generating learning models because many microservices used for autoscaling generate different metrics or logs in the same timestamp. In this study, we propose a cloud data refining module and structure that collects metric or log data in a microservice environment implemented by Kubernetes; and arranges it into computing resources corresponding to each service so that AI models can learn and analogize service-specific failures. We obtained Kubernetes-based AIOps learning data through this module, and after learning the built dataset through the AI model, we verified the prediction result through the differences between the obtained and actual data.

Face Detection by Eye Detection with Progressive Thresholding

  • Jung, Ji-Moon;Kim, Tae-Chul;Wie, Eun-Young;Nam, Ki-Gon
    • 제어로봇시스템학회:학술대회논문집
    • /
    • 2005.06a
    • /
    • pp.1689-1694
    • /
    • 2005
  • Face detection plays an important role in face recognition, video surveillance, and human computer interface. In this paper, we present a face detection system using eye detection with progressive thresholding from a digital camera. The face candidate is detected by using skin color segmentation in the YCbCr color space. The face candidates are verified by detecting the eyes that is located by iterative thresholding and correlation coefficients. Preprocessing includes histogram equalization, log transformation, and gray-scale morphology for the emphasized eyes image. The distance of the eye candidate points generated by the progressive increasing threshold value is employed to extract the facial region. The process of the face detection is repeated by using the increasing threshold value. Experimental results show that more enhanced face detection in real time.

  • PDF

Page Logging System for Web Mining Systems (웹마이닝 시스템을 위한 페이지 로깅 시스템)

  • Yun, Seon-Hui;O, Hae-Seok
    • The KIPS Transactions:PartC
    • /
    • v.8C no.6
    • /
    • pp.847-854
    • /
    • 2001
  • The Web continues to grow fast rate in both a large aclae volume of traffic and the size and complexity of Web sites. Along with growth, the complexity of tasks such as Web site design Web server design and of navigating simply through a Web site have increased. An important input to these design tasks is the analysis of how a web site is being used. The is paper proposes a Page logging System(PLS) identifying reliably user sessions required in Web mining system PLS consists of Page Logger acquiring all the page accesses of the user Log processor producing user session from these data, and statements to incorporate a call to page logger applet. Proposed PLS abbreviates several preprocessing tasks which spends a log of time and efforts that must be performed in Web mining systems. In particular, it simplifies the complexity of transaction identification phase through acquiring directly the amount of time a user stays on a page. Also PLS solves local cache hits and proxy IPs that create problems with identifying user sessions from Web sever log.

  • PDF