• Title/Summary/Keyword: log data

Search Result 2,120, Processing Time 0.026 seconds

Frequency Distribution Characteristics of Formation Density Derived from Log and Core Data throughout the Southern Korean Peninsula (남한지역 검층밀도 자료의 특성 분석)

  • Kim, Yeonghwa;Kim, Ki Hwan;Kim, Jongman;Hwang, Se Ho
    • The Journal of Engineering Geology
    • /
    • v.25 no.2
    • /
    • pp.281-290
    • /
    • 2015
  • Log density data were collected and compared with the core density data throughout the southern Korean Peninsula. The comparison reveals that the log densities obtained from gamma-gamma log are much lower than the core densities obtained from laboratory density measurement of core samples. The anomalously low log densities can be attributed to the small-source density log data. Correlation analysis reveals differences between densities derived from the two methods, indicating that a data quality problem arises when using small-source log data. The problem is probably due to the fact that small-source data have not been obtained under ideal conditions for maintaining the appropriate relationship between gamma response and formation density. The frequency distribution characteristics of formation density in the southern Korean Peninsula could be determined using the core and the standard-source log data which are well-correlated.

For Improving Security Log Big Data Analysis Efficiency, A Firewall Log Data Standard Format Proposed (보안로그 빅데이터 분석 효율성 향상을 위한 방화벽 로그 데이터 표준 포맷 제안)

  • Bae, Chun-sock;Goh, Sung-cheol
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.30 no.1
    • /
    • pp.157-167
    • /
    • 2020
  • The big data and artificial intelligence technology, which has provided the foundation for the recent 4th industrial revolution, has become a major driving force in business innovation across industries. In the field of information security, we are trying to develop and improve an intelligent security system by applying these techniques to large-scale log data, which has been difficult to find effective utilization methods before. The quality of security log big data, which is the basis of information security AI learning, is an important input factor that determines the performance of intelligent security system. However, the difference and complexity of log data by various product has a problem that requires excessive time and effort in preprocessing big data with poor data quality. In this study, we research and analyze the cases related to log data collection of various firewall. By proposing firewall log data collection format standard, we hope to contribute to the development of intelligent security systems based on security log big data.

Behavior analysis of entrance applicants using web log data (웹 로그데이터를 이용한 대학입시 지원자 행태 분석)

  • Choi, Seung-Bae;Kang, Chang-Wan;Cho, Jang-Sik
    • Journal of the Korean Data and Information Science Society
    • /
    • v.20 no.3
    • /
    • pp.493-504
    • /
    • 2009
  • The web log data analysis is to analysis traces which visitors remain while they drop by a web-site. Ultimately it can help to obtain a lot of useful information that can efficiently manage homepage and perform CRM(customer relationship management) using obtained information. In this paper, we provide a basic information to manage efficiently homepage of D university and to establish strategy for invitation of new pupil, as analyzing web log data for D university.

  • PDF

Design of Secure Log System in Cloud Computing Environment (클라우드 컴퓨팅 환경에서의 안전한 로그 시스템 설계)

  • Lee, Byung-Do;Shin, Sang Uk
    • Journal of Korea Multimedia Society
    • /
    • v.19 no.2
    • /
    • pp.300-307
    • /
    • 2016
  • Cloud computing that provide a elastic computing service is more complex compared to the existing computing systems. Accordingly, it has become increasingly important to maintain the stability and reliability of the computing system. And troubleshooting and real-time monitoring to address these challenges must be performed essentially. For these goals, the handling of the log data is needed, but this task in cloud computing environment may be more difficult compared to the traditional logging system. In addition, there are another challenges in order to have the admissibility of the collected log data in court. In this paper, we design secure logging service that provides the management and reliability of log data in a cloud computing environment and then analyze the proposed system.

A Pilot Study of the Scanning Beam Quality Assurance Using Machine Log Files in Proton Beam Therapy

  • Chung, Kwangzoo
    • Progress in Medical Physics
    • /
    • v.28 no.3
    • /
    • pp.129-133
    • /
    • 2017
  • The machine log files recorded by a scanning control unit in proton beam therapy system have been studied to be used as a quality assurance method of scanning beam deliveries. The accuracy of the data in the log files have been evaluated with a standard calibration beam scan pattern. The proton beam scan pattern has been delivered on a gafchromic film located at the isocenter plane of the proton beam treatment nozzle and found to agree within ${\pm}1.0mm$. The machine data accumulated for the scanning beam proton therapy of five different cases have been analyzed using a statistical method to estimate any systematic error in the data. The high-precision scanning beam log files in line scanning proton therapy system have been validated to be used for off-line scanning beam monitoring and thus as a patient-specific quality assurance method. The use of the machine log files for patient-specific quality assurance would simplify the quality assurance procedure with accurate scanning beam data.

UX Analysis for Mobile Devices Using MapReduce on Distributed Data Processing Platform (MapReduce 분산 데이터처리 플랫폼에 기반한 모바일 디바이스 UX 분석)

  • Kim, Sungsook;Kim, Seonggyu
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.2 no.9
    • /
    • pp.589-594
    • /
    • 2013
  • As the concept of web characteristics represented by openness and mind sharing grows more and more popular, device log data generated by both users and developers have become increasingly complicated. For such reasons, a log data processing mechanism that automatically produces meaningful data set from large amount of log records have become necessary for mobile device UX(User eXperience) analysis. In this paper, we define the attributes of to-be-analyzed log data that reflect the characteristics of a mobile device and collect real log data from mobile device users. Along with the MapReduce programming paradigm in Hadoop platform, we have performed a mobile device User eXperience analysis in a distributed processing environment using the collected real log data. We have then demonstrated the effectiveness of the proposed analysis mechanism by applying the various combinations of Map and Reduce steps to produce a simple data schema from the large amount of complex log records.

Anomaly Detection of Hadoop Log Data Using Moving Average and 3-Sigma (이동 평균과 3-시그마를 이용한 하둡 로그 데이터의 이상 탐지)

  • Son, Siwoon;Gil, Myeong-Seon;Moon, Yang-Sae;Won, Hee-Sun
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.6
    • /
    • pp.283-288
    • /
    • 2016
  • In recent years, there have been many research efforts on Big Data, and many companies developed a variety of relevant products. Accordingly, we are able to store and analyze a large volume of log data, which have been difficult to be handled in the traditional computing environment. To handle a large volume of log data, which rapidly occur in multiple servers, in this paper we design a new data storage architecture to efficiently analyze those big log data through Apache Hive. We then design and implement anomaly detection methods, which identify abnormal status of servers from log data, based on moving average and 3-sigma techniques. We also show effectiveness of the proposed detection methods by demonstrating that our methods identifies anomalies correctly. These results show that our anomaly detection is an excellent approach for properly detecting anomalies from Hadoop log data.

Likelihood based inference for the shape parameter of Pareto Distribution

  • Lee, Jae-Un;Lee, Woo-Dong
    • Journal of the Korean Data and Information Science Society
    • /
    • v.19 no.4
    • /
    • pp.1173-1181
    • /
    • 2008
  • In this paper, when the parameter of interest is the shape parameter in Pareto distribution, we develop likelihood based inference for this parameter. Specially, we develop signed log-likelihood ratio statistic and the modified signed log-likelihood ratio statistic for the shape parameter. It is well-known that as sample size grows, the modified signed log-likelihood ratio statistic converges to standard normal distribution faster than the signed log-likelihood ratio statistic. But the computation of the modified signed log-likelihood statistic is hard or even impossible when the sufficient statistics and the ancillary statistics are not clear. In this case, one can consider an approximation to the modified signed log-likelihood statistic. Specially, when the parameter of interest is informationally orthogonal to the nuisance parameters, we propose the approximate modified signed log-likelihood statistic. Through simulation, we investigate the performances of the proposed statistics with the signed log-likelihood statistic.

  • PDF

Energy Feature Normalization for Robust Speech Recognition in Noisy Environments

  • Lee, Yoon-Jae;Ko, Han-Seok
    • Speech Sciences
    • /
    • v.13 no.1
    • /
    • pp.129-139
    • /
    • 2006
  • In this paper, we propose two effective energy feature normalization methods for robust speech recognition in noisy environments. In the first method, we estimate the noise energy and remove it from the noisy speech energy. In the second method, we propose a modified algorithm for the Log-energy Dynamic Range Normalization (ERN) method. In the ERN method, the log energy of the training data in a clean environment is transformed into the log energy in noisy environments. If the minimum log energy of the test data is outside of a pre-defined range, the log energy of the test data is also transformed. Since the ERN method has several weaknesses, we propose a modified transform scheme designed to reduce the residual mismatch that it produces. In the evaluation conducted on the Aurora2.0 database, we obtained a significant performance improvement.

  • PDF

Small sample likelihood based inference for the normal variance ratio

  • Lee, Woo Dong
    • Journal of the Korean Data and Information Science Society
    • /
    • v.24 no.4
    • /
    • pp.911-918
    • /
    • 2013
  • This study deals with the small sample likelihood based inference for the ratio of two normal variances. The small sample likelihood inference is an approximation method. The signed log-likelihood ratio statistic and the modified signed log-likelihood ratio statistic, which converge to standard normal distribution, are proposed for the normal variance ratio. Through the simulation study, the coverage probabilities of confidence interval and power of the exact, the signed log-likelihood and the modified signed log-likelihood ratio statistic will be compared. A real data example will be provided.