• Title/Summary/Keyword: Log File Analysis

Search Result 61, Processing Time 0.02 seconds

Disjunctive Process Patterns Refinement and Probability Extraction from Workflow Logs

  • Kim, Kyoungsook;Ham, Seonghun;Ahn, Hyun;Kim, Kwanghoon Pio
    • Journal of Internet Computing and Services
    • /
    • v.20 no.3
    • /
    • pp.85-92
    • /
    • 2019
  • In this paper, we extract the quantitative relation data of activities from the workflow event log file recorded in the XES standard format and connect them to rediscover the workflow process model. Extract the workflow process patterns and proportions with the rediscovered model. There are four types of control-flow elements that should be used to extract workflow process patterns and portions with log files: linear (sequential) routing, disjunctive (selective) routing, conjunctive (parallel) routing, and iterative routing patterns. In this paper, we focus on four of the factors, disjunctive routing, and conjunctive path. A framework implemented by the authors' research group extracts and arranges the activity data from the log and converts the iteration of duplicate relationships into a quantitative value. Also, for accurate analysis, a parallel process is recorded in the log file based on execution time, and algorithms for finding and eliminating information distortion are designed and implemented. With these refined data, we rediscover the workflow process model following the relationship between the activities. This series of experiments are conducted using the Large Bank Transaction Process Model provided by 4TU and visualizes the experiment process and results.

A Model for Illegal File Access Tracking Using Windows Logs and Elastic Stack

  • Kim, Jisun;Jo, Eulhan;Lee, Sungwon;Cho, Taenam
    • Journal of Information Processing Systems
    • /
    • v.17 no.4
    • /
    • pp.772-786
    • /
    • 2021
  • The process of tracking suspicious behavior manually on a system and gathering evidence are labor-intensive, variable, and experience-dependent. The system logs are the most important sources for evidences in this process. However, in the Microsoft Windows operating system, the action events are irregular and the log structure is difficult to audit. In this paper, we propose a model that overcomes these problems and efficiently analyzes Microsoft Windows logs. The proposed model extracts lists of both common and key events from the Microsoft Windows logs to determine detailed actions. In addition, we show an approach based on the proposed model applied to track illegal file access. The proposed approach employs three-step tracking templates using Elastic Stack as well as key-event, common-event lists and identify event lists, which enables visualization of the data for analysis. Using the three-step model, analysts can adjust the depth of their analysis.

HBase based Business Process Event Log Schema Design of Hadoop Framework

  • Ham, Seonghun;Ahn, Hyun;Kim, Kwanghoon Pio
    • Journal of Internet Computing and Services
    • /
    • v.20 no.5
    • /
    • pp.49-55
    • /
    • 2019
  • Organizations design and operate business process models to achieve their goals efficiently and systematically. With the advancement of IT technology, the number of items that computer systems can participate in and the process becomes huge and complicated. This phenomenon created a more complex and subdivide flow of business process.The process instances that contain workcase and events are larger and have more data. This is an essential resource for process mining and is used directly in model discovery, analysis, and improvement of processes. This event log is getting bigger and broader, which leads to problems such as capacity management and I / O load in management of existing row level program or management through a relational database. In this paper, as the event log becomes big data, we have found the problem of management limit based on the existing original file or relational database. Design and apply schemes to archive and analyze large event logs through Hadoop, an open source distributed file system, and HBase, a NoSQL database system.

An Accurate Log Object Recognition Technique

  • Jiho, Ju;Byungchul, Tak
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.2
    • /
    • pp.89-97
    • /
    • 2023
  • In this paper, we propose factors that make log analysis difficult and design technique for detecting various objects embedded in the logs which helps in the subsequent analysis. In today's IT systems, logs have become a critical source data for many advanced AI analysis techniques. Although logs contain wealth of useful information, it is difficult to directly apply techniques since logs are semi-structured by nature. The factors that interfere with log analysis are various objects such as file path, identifiers, JSON documents, etc. We have designed a BERT-based object pattern recognition algorithm for these objects and performed object identification. Object pattern recognition algorithms are based on object definition, GROK pattern, and regular expression. We find that simple pattern matchings based on known patterns and regular expressions are ineffective. The results show significantly better accuracy than using only the patterns and regular expressions. In addition, in the case of the BERT model, the accuracy of classifying objects reached as high as 99%.

An Efficient Design and Implementation of an MdbULPS in a Cloud-Computing Environment

  • Kim, Myoungjin;Cui, Yun;Lee, Hanku
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.9 no.8
    • /
    • pp.3182-3202
    • /
    • 2015
  • Flexibly expanding the storage capacity required to process a large amount of rapidly increasing unstructured log data is difficult in a conventional computing environment. In addition, implementing a log processing system providing features that categorize and analyze unstructured log data is extremely difficult. To overcome such limitations, we propose and design a MongoDB-based unstructured log processing system (MdbULPS) for collecting, categorizing, and analyzing log data generated from banks. The proposed system includes a Hadoop-based analysis module for reliable parallel-distributed processing of massive log data. Furthermore, because the Hadoop distributed file system (HDFS) stores data by generating replicas of collected log data in block units, the proposed system offers automatic system recovery against system failures and data loss. Finally, by establishing a distributed database using the NoSQL-based MongoDB, the proposed system provides methods of effectively processing unstructured log data. To evaluate the proposed system, we conducted three different performance tests on a local test bed including twelve nodes: comparing our system with a MySQL-based approach, comparing it with an Hbase-based approach, and changing the chunk size option. From the experiments, we found that our system showed better performance in processing unstructured log data.

CERES: A Log-based, Interactive Web Analytics System for Backbone Networks (CERES: 백본망 로그 기반 대화형 웹 분석 시스템)

  • Suh, Ilhyun;Chung, Yon Dohn
    • KIISE Transactions on Computing Practices
    • /
    • v.21 no.10
    • /
    • pp.651-657
    • /
    • 2015
  • The amount of web traffic has increased as a result of the rapid growth of the use of web-based applications. In order to obtain valuable information from web logs, we need to develop systems that can support interactive, flexible, and efficient ways to analyze and handle large amounts of data. In this paper, we present CERES, a log-based, interactive web analytics system for backbone networks. Since CERES focuses on analyzing web log records generated from backbone networks, it is possible to perform a web analysis from the perspective of a network. CERES is designed for deployment in a server cluster using the Hadoop Distributed File System (HDFS) as the underlying storage. We transform and store web log records from backbone networks into relations and then allow users to use a SQL-like language to analyze web log records in a flexible and interactive manner. In particular, we use the data cube technique to enable the efficient statistical analysis of web log. The system provides users a web-based, multi-modal user interface.

Design and Implementation of an Interestingness Analysis System for Web Personalizatoion & Customization

  • Jung, Youn-Hong;Kim, I-I;Park, Kyoo-seok
    • Journal of Korea Multimedia Society
    • /
    • v.6 no.4
    • /
    • pp.707-713
    • /
    • 2003
  • Convenience and promptness of the internet have been not only making the electronic commerce grow rapidly in case of website, analyzing a navigation pattern of the users has been also making personalization and customization techniques develop rapidly for providing service accordant to individual interestingness. Web personalization and customization skill has been utilizing various methods, such as web log mining to use web log data and web mining to use the transaction of users etc, especially e-CRM analyzing a navigation pattern of the users. In this paper, We measure exact duration time of the users in web page and web site, compute weight about duration time each page, and propose a way to comprehend e-loyalty through the computed weight.

  • PDF

A Study on Usability Evaluation for Improving Quality of User Services in CNU Digital Library Website (이용자 서비스의 품질 향상을 위한 웹사이트 사용성 평가에 관한 연구)

  • Lee, Eung-Bong
    • Journal of the Korean Society for Library and Information Science
    • /
    • v.36 no.4
    • /
    • pp.311-329
    • /
    • 2002
  • This study aims to evaluate the usability of XML-based Chungnam National University Digital Library Website. Usability testing are used three method, that is Log File Analysis, Online Questionnaire and Heuristic Evaluation. Log files are analyzed in using WiseLog Enterprise and Hit Analyzer for Enterprise jx 1.5 Beta Version and total 21 items are analyzed and evaluated in using online questionnaire method and heuristic evaluation is inquired by experts of web and information retrieval fields. Finally this paper was suggested some problems and improvements to advance quality of user services in university digital library website in korea.

Implementation of Mobile Web Server Log Analyzer (모바일 웹 서버 로그 분석기 구현)

  • Park Hong-Jin
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.11 no.2
    • /
    • pp.63-68
    • /
    • 2006
  • If web server occurs my fault, log file is important information to find out occurred error. The web server analyzer plays a vital role in improving the web service quality by analyzing log files stores in the web server. But, most conventional web server analysis programs are provided over wired networks, there are problems to be dealt with in terms of time-space restrictions and mobility. In order to solve above problem, this paper represent web server analyser based mobile. In case web server manager based mobile, it can immediately know recent log information accessing real-rime as also anytime, anywhere.

  • PDF

Navigational Structure and User Behavior Modeling for Restructuring of Web-based Information Systems (웹기반 정보시스템의 재구성을 위한 항해구조 및 사용자행동 모델링)

  • 박학수;황성하;이강수
    • Journal of Korea Multimedia Society
    • /
    • v.5 no.6
    • /
    • pp.730-744
    • /
    • 2002
  • A Web-Based Information System(WBIS), a typical structure of recently information systems, should be dynamically restructured in order to satisfy user's need and make a profit. Thus, we should analyze and modelize the navigational structure of WBIS and utilize it by modeling the navigational structure of behavior of user through log-file as system restructuring. In this paper, we propose the modeling method for navigational structure and user behavior to restructure WBIS including shopping mall. Also, we suggest the structural model, state transition model, Petri net model and analysis method and analyze and implement modeling algorithm for user behavior to analyze log-file of it. Then, we propose some restructuring heuristic and apply the methods to the example of WBIS.

  • PDF