• Title/Summary/Keyword: anomalous data

Search Result 209, Processing Time 0.021 seconds

A Comparative Analysis of Linearity and Range of Gravity and Magnetic Data Using Variogram (베리오그램을 이용한 중력과 자력 자료의 선형성 및 상관거리 비교 분석)

  • Park, Gye-soon;Park, No-Wook
    • Journal of the Korean earth science society
    • /
    • v.31 no.2
    • /
    • pp.119-128
    • /
    • 2010
  • To make reliable interpretations on the sparse spatial data, the spatial distribution characteristics that are inevitable for spatial estimation should be properly analyzed. Variograms have been widely used for obtaining the spatial characteristics inherent to data in spatial estimation problems. But their applications were limited as the basic information for further data estimation. Therefore, the additional analysis of the meaning of variograms is required for more reliable data processing and interpretations. In this paper, we investigated the proper meaning of variogram values and the specific features of distributions which can be obtained through variogram analysis. Variograms can provide the information on both linearity and the strength changes of interrelationships between the data sets according to the direction and lag distance. First, sill and range values, which are main parameters of variograms, were analyzed. Then a similarity range using spatial auto-correlation values was introduced to verify the applicability of linearity analysis through the comparative study of spatial distribution features of gravity and magnetic data collected in Hwasan caldera. Through these analyses, we were able to identify the dissimilar patterns of gravity and magnetic data that became apparent according to the distribution and variation ranges of the data sets. It is inferred that the gravity and magnetic anomalous bodies are extended to the ground because linearity direction of gravity and magnetic data appear similarly with linearity derection of topography in Hwasan caldera.

Comparative analysis of wavelet transform and machine learning approaches for noise reduction in water level data (웨이블릿 변환과 기계 학습 접근법을 이용한 수위 데이터의 노이즈 제거 비교 분석)

  • Hwang, Yukwan;Lim, Kyoung Jae;Kim, Jonggun;Shin, Minhwan;Park, Youn Shik;Shin, Yongchul;Ji, Bongjun
    • Journal of Korea Water Resources Association
    • /
    • v.57 no.3
    • /
    • pp.209-223
    • /
    • 2024
  • In the context of the fourth industrial revolution, data-driven decision-making has increasingly become pivotal. However, the integrity of data analysis is compromised if data quality is not adequately ensured, potentially leading to biased interpretations. This is particularly critical for water level data, essential for water resource management, which often encounters quality issues such as missing values, spikes, and noise. This study addresses the challenge of noise-induced data quality deterioration, which complicates trend analysis and may produce anomalous outliers. To mitigate this issue, we propose a noise removal strategy employing Wavelet Transform, a technique renowned for its efficacy in signal processing and noise elimination. The advantage of Wavelet Transform lies in its operational efficiency - it reduces both time and costs as it obviates the need for acquiring the true values of collected data. This study conducted a comparative performance evaluation between our Wavelet Transform-based approach and the Denoising Autoencoder, a prominent machine learning method for noise reduction.. The findings demonstrate that the Coiflets wavelet function outperforms the Denoising Autoencoder across various metrics, including Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), and Mean Squared Error (MSE). The superiority of the Coiflets function suggests that selecting an appropriate wavelet function tailored to the specific application environment can effectively address data quality issues caused by noise. This study underscores the potential of Wavelet Transform as a robust tool for enhancing the quality of water level data, thereby contributing to the reliability of water resource management decisions.

Advanced protocol against MITM attacks in Industrial Control System (산업제어시스템에서의 MITM 공격을 방어하기 위해 개선된 프로토콜)

  • Ko, Moo-seong;Oh, Sang-kyo;Lee, Kyung-ho
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.25 no.6
    • /
    • pp.1455-1463
    • /
    • 2015
  • If the industrial control system is infected by malicious worm such as Stuxnet, national disaster could be caused inevitably. Therefore, most of the industrial control system defence is focused on intrusion detection in network to protect against these threats. Conventional method is effective to monitor network traffic and detect anomalous patterns, but normal traffic pattern attacks using MITM technique are difficult to be detected. This study analyzes the PROFINET/DCP protocol and weaknesses with the data collected in real industrial control system. And add the authentication data field to secure the protocol, find out the applicability. Improved protocol may prevent the national disaster and defend against MITM attacks.

Machine Learning based on Approach for Classification of Abnormal Data in Shop-floor (제조 현장의 비정상 데이터 분류를 위한 기계학습 기반 접근 방안 연구)

  • Shin, Hyun-Juni;Oh, Chang-Heon
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.21 no.11
    • /
    • pp.2037-2042
    • /
    • 2017
  • The manufacturing facility is generally operated by a pre-set program under the existing factory automation system. On the other hand, the manufacturing facility must decide how to operate autonomously in Industry 4.0. Determining the operation mode of the production facility itself means, for example, that it detects the abnormality such as the deterioration of the facility at the shop-floor, prediction of the occurrence of the problem, detection of the defect of the product, In this paper, we propose a manufacturing process modeling using a queue for detection of manufacturing process abnormalities at the shop-floor, and detect abnormalities in the modeling using SVM, one of the machine learning techniques. The queue was used for M / D / 1 and the conveyor belt manufacturing system was modeled based on ${\mu}$, ${\lambda}$, and ${\rho}$. SVM was used to detect anomalous signs through changes in ${\rho}$.

Interannual Variability of Sea Water Temperatures in the Southern Waters of the Korean East Sea (한국 동남해역의 장주기 수온변동)

  • Ro, Young Jae
    • 한국해양학회지
    • /
    • v.24 no.1
    • /
    • pp.1-14
    • /
    • 1989
  • This study analyzes the interannual periodicity by using the statistical techniques of probability, spectral analysis, empirical orthogonal function analysis (EOF), and coherency analysis. The data base for this study is the time series of 1971-1985 temperature, salinity in the southern waters of the East Sea, 1960-1986 mean sea level at Pusan and Izuhara, and 1960-1986 sea level atmospheric pressure at Pusan. The appearances of anomalous temperatures higher and lower than 15-year mean monthly average with one standard deviation are about 30% of total data. The significant interannual period for temperature, salinity and sea level fluctuation is 36.6, and 23.3 months. The empirical orthogonal function analyses show that the 1st mode of the EOFs is responsible for more than 90% of total variance of the surface temperature variations, while in near-bottom waters, the relative importance of the higher EOF modes is much greater explaining more than 30% of total variance. The coherency between normalized temperatures and salinities is significant at the interannual period of 36.6 and 21.3 months.

  • PDF

Development of the Impulse Response Measurement System for Non-destructive Test of Slab Structure (판상 구조물 비파괴검사를 위한 충격응답시험기의 개발)

  • Chung, Hojoon;Lee, Heuisoon;Oh, Seokhoon;Song, Sung-Ho
    • Geophysics and Geophysical Exploration
    • /
    • v.16 no.1
    • /
    • pp.45-52
    • /
    • 2013
  • We developed a Impulse Response Measurement System, including hardware system and data analysis software, for non-destructive test of slab structure. And, we carried out impulse response measurements on the pavement to test performance of the system. In the field test, the developed system measured impulse response stably and showed parameters immediately. Test results showed that dynamic stiffness and average mobility varies significantly depending on the characteristics of the pavement materials. Some data showed anomalous values those reflect variations in pavement itself or subsurface ground. Developed system gives informations of conditions of slab structure easily and quickly. So, 2-D monitoring with the system will be helpful in maintaining various slab structures.

The Recent Increase in the Heavy Rainfall Events in August over the Korean Peninsula

  • Cha, Eun-Jeong;Kimoto, Masahide;Lee, Eun-Jeong;Jhun, Jong-Ghap
    • Journal of the Korean earth science society
    • /
    • v.28 no.5
    • /
    • pp.585-597
    • /
    • 2007
  • The characteristics of the rainfall events on the Korean peninsula have been investigated by means of regional and global observational data collected from 1954 to 2004 with an emphasis on extreme cases $80\;mm\;day^{-1}$. According to our analysis, long-term annual rainfall anomalies show an increasing trend. This trend is pronounced in the month of August, when both the amount of monthly rainfall and the frequency of extreme events increase significantly. Composite maps on August during the 8 wet years reveal warm SST anomalies over the eastern Philippine Sea which are associated with enhanced convection and vertical motion and intensified positive SLP over central Eurasia during August. The rainfall pattern suggests that the most significant increase in moisture supply over the southern parts of China and Korea in August is associated with positive SLP changes over Eurasia and negative SLP changes over the subtropical western Pacific off the east coast of south China. The frequent generation of typhoons over the warm eastern Philippine Sea and their tracks appear to influence the extreme rainfall events in Korea during the month of August. The typhoons in August mainly passed the western coast of Korea, resulting in the frequent occurrence of extreme rainfall events in this region. Furthermore, anomalous cyclonic circulations over the eastern Philippine Sea also promoted the generation of tropical cyclones. The position of pressure systems - positive SLP over Eurasia and negative SLP over the subtropical Pacific - in turn provided a pathway for typhoons. The moisture is then effectively transported further north toward Korea and east toward the southern parts of China during the extreme rainfall period.

Modified Electrical Resistivity Survey and its Interpretation for Leakage Path Detection of Water Facilities (수변구조물의 누수 경로 탐지를 위한 변형된 전기비저항 탐사 및 자료 해석)

  • Lee, Bomi;Oh, Seokhoon
    • Geophysics and Geophysical Exploration
    • /
    • v.19 no.4
    • /
    • pp.200-211
    • /
    • 2016
  • To support cross potential array and direct potential array, the array for leakage detection of all kinds of water facilities is proposed and it is named as the D-Lux array. The D-Lux array data are arranged to a coloured matrix and it is called the D-Lux view. Low potential difference of anomalous zone shown in D-Lux view implies the indication of leakage zone. Furthermore, for an intuitive interpretation of D-Lux array, equipotential distribution map is made by using D-Lux and direct potential array data. Equipotential distribution map makes us possible to predict import point, export point and the path of water leakage that we could have not anticipated in D-Lux view and the graphs. The water tank experiment and numerical analysis were carried out as preparatory experiment and the field explorations were conducted at a concrete weir and a fill dam. As a result, effective and specific detection of leakage path was possible for the concrete weir and the fill dam.

GIS/GPS based Precision Agriculture Model in India -A Case study

  • Mudda, Suresh Kumar
    • Agribusiness and Information Management
    • /
    • v.10 no.2
    • /
    • pp.1-7
    • /
    • 2018
  • In the present day context of changing information needs of the farmers and diversified production systems there is an urgent need to look for the effective extension support system for the small and marginal farmers in the developing countries like India. The rapid developments in the collection and analysis of field data by using the spatial technologies like GPS&GIS were made available for the extension functionaries and clientele for the diversified information needs. This article describes the GIS and GPS based decision support system in precision agriculture for the resource poor farmers. Precision farming techniques are employed to increase yield, reduce production costs, and minimize negative impacts to the environment. The parameters those can affect the crop yields, anomalous factors and variations in management practices can be evaluated through this GPS and GIS based applications. The spatial visualisation capabilities of GIS technology interfaced with a relational database provide an effective method for analysing and displaying the impacts of Extension education and outreach projects for small and marginal farmers in precision agriculture. This approach mainly benefits from the emergence and convergence of several technologies, including the Global Positioning System (GPS), geographic information system (GIS), miniaturised computer components, automatic control, in-field and remote sensing, mobile computing, advanced information processing, and telecommunications. The PPP convergence of person (farmer), project (the operational field) and pixel (the digital images related to the field and the crop grown in the field) will better be addressed by this decision support model. So the convergence and emergence of such information will further pave the way for categorisation and grouping of the production systems for the better extension delivery. In a big country like India where the farmers and holdings are many in number and diversified categorically such grouping is inevitable and also economical. With this premise an attempt has been made to develop a precision farming model suitable for the developing countries like India.

Novelty Detection on Web-server Log Dataset (웹서버 로그 데이터의 이상상태 탐지 기법)

  • Lee, Hwaseong;Kim, Ki Su
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.23 no.10
    • /
    • pp.1311-1319
    • /
    • 2019
  • Currently, the web environment is a commonly used area for sharing information and conducting business. It is becoming an attack point for external hacking targeting on personal information leakage or system failure. Conventional signature-based detection is used in cyber threat but signature-based detection has a limitation that it is difficult to detect the pattern when it is changed like polymorphism. In particular, injection attack is known to the most critical security risks based on web vulnerabilities and various variants are possible at any time. In this paper, we propose a novelty detection technique to detect abnormal state that deviates from the normal state on web-server log dataset(WSLD). The proposed method is a machine learning-based technique to detect a minor anomalous data that tends to be different from a large number of normal data after replacing strings in web-server log dataset with vectors using machine learning-based embedding algorithm.