• Title/Summary/Keyword: Web Log Data

Search Result 176, Processing Time 0.026 seconds

OLAP System and Performance Evaluation for Analyzing Web Log Data (웹 로그 분석을 위한 OLAP 시스템 및 성능 평가)

  • 김지현;용환승
    • Journal of Korea Multimedia Society
    • /
    • v.6 no.5
    • /
    • pp.909-920
    • /
    • 2003
  • Nowadays, IT for CRM has been growing and developed rapidly. Typical techniques are statistical analysis tools, on-line multidimensional analytical processing (OLAP) tools, and data mining algorithms (such neural networks, decision trees, and association rules). Among customer data, web log data is very important and to use these data efficiently, applying OLAP technology to analyze multi-dimensionally. To make OLAP cube, we have to precalculate multidimensional summary results in order to get fast response. But as the number of dimensions and sparse cells increases, data explosion occurs seriously and the performance of OLAP decreases. In this paper, we presented why the web log data sparsity occurs and then what kinds of sparsity patterns generate in the two and t.he three dimensions for OLAP. Based on this research, we set up the multidimensional data models and query models for benchmark with each sparsity patterns. Finally, we evaluated the performance of three OLAP systems (MS SQL 2000 Analysis Service, Oracle Express and C-MOLAP).

  • PDF

Web Attack Classification via WAF Log Analysis: AutoML, CNN, RNN, ALBERT (웹 방화벽 로그 분석을 통한 공격 분류: AutoML, CNN, RNN, ALBERT)

  • Youngbok Jo;Jaewoo Park;Mee Lan Han
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.34 no.4
    • /
    • pp.587-596
    • /
    • 2024
  • Cyber Attack and Cyber Threat are getting confused and evolved. Therefore, using AI(Artificial Intelligence), which is the most important technology in Fourth Industry Revolution, to build a Cyber Threat Detection System is getting important. Especially, Government's SOC(Security Operation Center) is highly interested in using AI to build SOAR(Security Orchestration, Automation and Response) Solution to predict and build CTI(Cyber Threat Intelligence). In this thesis, We introduce the Cyber Threat Detection System by analyzing Network Traffic and Web Application Firewall(WAF) Log data. Additionally, we apply the well-known TF-IDF(Term Frequency-Inverse Document Frequency) method and AutoML technology to classify Web traffic attack type.

Web Log Analysis Using Support Vector Regression

  • Jun, Sung-Hae;Lim, Min-Taik;Jorn, Hong-Seok;Hwang, Jin-Soo;Park, Seong-Yong;Kim, Jee-Yun;Oh, Kyung-Whan
    • Communications for Statistical Applications and Methods
    • /
    • v.10 no.1
    • /
    • pp.61-77
    • /
    • 2003
  • Due to the wide expansion of the internet, people can freely get information what they want with lesser efforts. However without adequate forms or rules to follow, it is getting more and more difficult to get necessary information. Because of seemingly chaotic status of the current web environment, it is sometimes called "Dizzy web" The user should wander from page to page to get necessary information. Therefore we need to construct system which properly recommends appropriate information for general user. The representative research field for this system is called Recommendation System(RS), The collaborative recommendation system is one of the RS. It was known to perform better than the other systems. When we perform the web user modeling or other web-mining tasks, the continuous feedback data is very important and frequently used. In this paper, we propose a collaborative recommendation system which can deal with the continuous feedback data and tried to construct the web page prediction system. We use a sojourn time of a user as continuous feedback data and combine the traditional model-based algorithm framework with the Support Vector Regression technique. In our experiments, we show the accuracy of our system and the computing time of page prediction compared with Pearson's correlation algorithm.algorithm.

Similarity Pattern Analysis of Web Log Data using Multidimensional FCM (다차원 FCM을 이용한 웹 로그 데이터의 유사 패턴 분석)

  • 김미라;조동섭
    • Proceedings of the Korean Information Science Society Conference
    • /
    • 2002.10d
    • /
    • pp.190-192
    • /
    • 2002
  • 데이터 마이닝(Data Mining)이란 저장된 많은 양의 자료로부터 통계적 수학적 분석방법을 이용하여 다양한 가치 있는 정보를 찾아내는 일련의 과정이다. 데이터 클러스터링은 이러한 데이터 마이닝을 위한 하나의 중요한 기법이다. 본 논문에서는 Fuzzy C-Means 알고리즘을 이용하여 웹 사용자들의 행위가 기록되어 있는 웹 로그 데이터를 데이터 클러스터링 하는 방법에 관하여 연구하고자 한다. Fuzzv C-Means 클러스터링 알고리즘은 각 데이터와 각 클러스터 중심과의 거리를 고려한 유사도 측정에 기초한 목적 함수의 최적화 방식을 사용한다. 웹 로그 데이터의 여러 필드 중에서 사용자 IP, 시간, 웹 페이지 필드를 WLDF(Web Log Data for FCM)으로 가공한 후, 다차원 Fuzzy C-Means 클러스터링을 한다. 그리고 이를 이용하여 샘플 데이터와 임의의 데이터간의 유사 패턴 분석을 하고자 한다.

  • PDF

Distributed FTP Server for Log Mining System on ACE (분산 FTP 서버의 ACE 기반 로그 마이닝 시스템)

  • Min, Su-Hong;Cho, Dong-Sub
    • Proceedings of the KIEE Conference
    • /
    • 2002.11c
    • /
    • pp.465-468
    • /
    • 2002
  • Today large corporations are constructing distributed server environment. Many corporations are respectively operating Web server, FTP server, Mail server and DB server on heterogeneous operation. However, there is the problem that a manager must manage each server individually. In this paper, we present distributed FTP server for log mining system on ACE. Proposed log mining system is based upon ACE (Adaptive Communication Environment) framework and data mining techniques. This system provides a united operation with distributed FTP server.

  • PDF

Fuzzy category based transaction analysis for web usage mining (웹 사용 마이닝을 위한 퍼지 카테고리 기반의 트랜잭션 분석 기법)

  • 이시헌;이지형
    • Proceedings of the Korean Institute of Intelligent Systems Conference
    • /
    • 2004.04a
    • /
    • pp.341-344
    • /
    • 2004
  • 웹 사용 마이닝(Web usage mining)은 웹 로그 파일(web log file)이나 웹 사용 데이터(Web usage data)에서 의미 있는 정보를 찾아내는 연구 분야이다. 웹 사용 마이닝에서 일반적으로 많이 사용하는 웹 로그 파일은 사용자들이 참조한 페이지의 단순한 리스트들이다. 따라서 단순히 웹 로그 파일만을 이용하는 방법만으로는 사용자가 참조했던 페이지의 내용을 반영하여 분석하는데에는 한계가 있다. 이러한 점을 개선하고자 본 논문에서는 페이지 위주가 아닌 웹 페이지가 포함하고 있는 내용(아이템)을 고려하는 새로운 퍼지 카테고리 기반의 웹 사용 마이닝 기법을 제시한다. 또한 사용자를 잘 파악하기 위해서 시간에 따라 관심의 변화를 파악하는 방법을 제시한다.

  • PDF

UX Analysis for Mobile Devices Using MapReduce on Distributed Data Processing Platform (MapReduce 분산 데이터처리 플랫폼에 기반한 모바일 디바이스 UX 분석)

  • Kim, Sungsook;Kim, Seonggyu
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.9
    • /
    • pp.589-594
    • /
    • 2013
  • As the concept of web characteristics represented by openness and mind sharing grows more and more popular, device log data generated by both users and developers have become increasingly complicated. For such reasons, a log data processing mechanism that automatically produces meaningful data set from large amount of log records have become necessary for mobile device UX(User eXperience) analysis. In this paper, we define the attributes of to-be-analyzed log data that reflect the characteristics of a mobile device and collect real log data from mobile device users. Along with the MapReduce programming paradigm in Hadoop platform, we have performed a mobile device User eXperience analysis in a distributed processing environment using the collected real log data. We have then demonstrated the effectiveness of the proposed analysis mechanism by applying the various combinations of Map and Reduce steps to produce a simple data schema from the large amount of complex log records.

A Data Mining Technique for Customer Behavior Association Analysis in Cyber Shopping Malls (가상상점에서 고객 행위 연관성 분석을 위한 데이터 마이닝 기법)

  • 김종우;이병헌;이경미;한재룡;강태근;유관종
    • The Journal of Society for e-Business Studies
    • /
    • v.4 no.1
    • /
    • pp.21-36
    • /
    • 1999
  • Using user monitoring techniques on web, marketing decision makers in cyber shopping malls can gather customer behavior data as well as sales transaction data and customer profiles. In this paper, we present a marketing rule extraction technique for customer behavior analysis in cyber shopping malls, The technique is an application of market basket analysis which is a representative data mining technique for extracting association rules. The market basket analysis technique is applied on a customer behavior log table, which provide association rules about web pages in a cyber shopping mall. The extracted association rules can be used for mall layout design, product packaging, web page link design, and product recommendation. A prototype cyber shopping mall with customer monitoring features and a customer behavior analysis algorithm is implemented using Java Web Server, Servlet, JDBC(Java Database Connectivity), and relational database on windows NT.

  • PDF

Real-Time Visualization of Web Usage Patterns and Anomalous Sessions (실시간 웹 사용 현황과 이상 행위에 대한 시각화)

  • 이병희;조상현;차성덕
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.14 no.4
    • /
    • pp.97-110
    • /
    • 2004
  • As modem web services become enormously complex, web attacks has become frequent and serious. Existing security solutions such as firewalls or signature-based intrusion detection systems are generally inadequate in securing web services, and analysis of raw web log data is simply impractical for most organizations. Visual display of "interpreted" web logs, with emphasis on anomalous web requests, is essential for an organization to efficiently track web usage patterns and detect possible web attacks. In this paper, we discuss various issues related to effective real-time visualization of web usage patterns and anomalies. We implemented a software tool named SAD (session anomaly detection) Viewer to satisfy such need and conducted an empirical study in which anomalous web traffics such as Misuse attacks, DoS attacks, Code-Red worms and Whisker scans were injected. Our study confirms that SAD Viewer is useful in assisting web security engineers to monitor web usage patterns in general and anomalous web sessions in particular.articular.

Implementation of Group Management System with Smart Phone Devices and Wireless Sensor Network (스마트폰 및 무선 센서 네트워크를 기반으로 한 그룹관리 시스템의 구현)

  • Lee, Seung-Joon;Jung, Kyung-Kwon;Lee, Hyun-Kwan;Eom, Ki-Hwan
    • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
    • /
    • 2011.05a
    • /
    • pp.378-381
    • /
    • 2011
  • The group management system with Wireless Sensor Network and android application is proposed in this paper. The proposed system was composed of personal devices with sensor nodes of WSN, manager device of android platform, and the web server. The sensor node used by each group member send a data packet to the manager device every 2 seconds. The leader device displays and transmits entire information to the web server. The web server represents these information through web page. Therefore, guardians can assure their group member's safety and security on the web page. The RSSI value of each sensor node converted by computed log-normal path loss model into distance value and displays on the manager device and the web page.

  • PDF