• Title/Summary/Keyword: Anomaly data detection

Search Result 394, Processing Time 0.024 seconds

Rank Correlation Coefficient of Energy Data for Identification of Abnormal Sensors in Buildings (에너지 데이터의 순위상관계수 기반 건물 내 오작동 기기 탐지)

  • Kim, Naeon;Jeong, Sihyun;Jang, Boyeon;Kim, Chong-Kwon
    • Journal of KIISE
    • /
    • v.44 no.4
    • /
    • pp.417-422
    • /
    • 2017
  • Anomaly detection is the identification of data that do not conform to a normal pattern or behavior model in a dataset. It can be utilized for detecting errors among data generated by devices or user behavior change in a social network data set. In this study, we proposed a new approach using rank correlation coefficient to efficiently detect abnormal data in devices of a building. With the increased push for energy conservation, many energy efficiency solutions have been proposed over the years. HVAC (Heating, Ventilating and Air Conditioning) system monitors and manages thousands of sensors such as thermostats, air conditioners, and lighting in large buildings. Currently, operators use the building's HVAC system for controlling efficient energy consumption. By using the proposed approach, it is possible to observe changes of ranking relationship between the devices in HVAC system and identify abnormal behavior in social network.

FLORA: Fuzzy Logic - Objective Risk Analysis for Intrusion Detection and Prevention

  • Alwi M Bamhdi
    • International Journal of Computer Science & Network Security
    • /
    • v.23 no.5
    • /
    • pp.179-192
    • /
    • 2023
  • The widespread use of Cloud Computing, Internet of Things (IoT), and social media in the Information Communication Technology (ICT) field has resulted in continuous and unavoidable cyber-attacks on users and critical infrastructures worldwide. Traditional security measures such as firewalls and encryption systems are not effective in countering these sophisticated cyber-attacks. Therefore, Intrusion Detection and Prevention Systems (IDPS) are necessary to reduce the risk to an absolute minimum. Although IDPSs can detect various types of cyber-attacks with high accuracy, their performance is limited by a high false alarm rate. This study proposes a new technique called Fuzzy Logic - Objective Risk Analysis (FLORA) that can significantly reduce false positive alarm rates and maintain a high level of security against serious cyber-attacks. The FLORA model has a high fuzzy accuracy rate of 90.11% and can predict vulnerabilities with a high level of certainty. It also has a mechanism for monitoring and recording digital forensic evidence which can be used in legal prosecution proceedings in different jurisdictions.

A Network Intrusion Security Detection Method Using BiLSTM-CNN in Big Data Environment

  • Hong Wang
    • Journal of Information Processing Systems
    • /
    • v.19 no.5
    • /
    • pp.688-701
    • /
    • 2023
  • The conventional methods of network intrusion detection system (NIDS) cannot measure the trend of intrusiondetection targets effectively, which lead to low detection accuracy. In this study, a NIDS method which based on a deep neural network in a big-data environment is proposed. Firstly, the entire framework of the NIDS model is constructed in two stages. Feature reduction and anomaly probability output are used at the core of the two stages. Subsequently, a convolutional neural network, which encompasses a down sampling layer and a characteristic extractor consist of a convolution layer, the correlation of inputs is realized by introducing bidirectional long short-term memory. Finally, after the convolution layer, a pooling layer is added to sample the required features according to different sampling rules, which promotes the overall performance of the NIDS model. The proposed NIDS method and three other methods are compared, and it is broken down under the conditions of the two databases through simulation experiments. The results demonstrate that the proposed model is superior to the other three methods of NIDS in two databases, in terms of precision, accuracy, F1- score, and recall, which are 91.64%, 93.35%, 92.25%, and 91.87%, respectively. The proposed algorithm is significant for improving the accuracy of NIDS.

Interpretation of Airborne Magnetic and Radioactive Data for the Uranium Deposit in Geumsan Area (금산 함우라늄광상 분포지역의 항공자력/방사능 탐사자료 해석)

  • Shin, Eun-Ju;Ko, Kwangbeom;An, Dongkuk;Han, Kyeongsoo
    • Geophysics and Geophysical Exploration
    • /
    • v.16 no.1
    • /
    • pp.36-44
    • /
    • 2013
  • We conducted the airborne magnetic and radiometric survey for the characterization of the black shale related and pyrometamorphic uranium deposits distributed in Geumsan area. For the successful characterization of the uranium deposits, the general geological and structural geological features were investigated based on the lithological and linear feature analysis to individual magnetic and radiometric data as the first step. Lithological analysis from the magnetic reduction to the pole and downward continuation map revealed that prominent positive anomalies caused by black and dark gray slate member were clearly recognized as magnetic sources. These results indicate that magnetic survey, even though it is not a direct method for the detection of uranium, can be a useful tool in uranium detection. By the linear feature analysis based on 2nd vertical derivative and curvature map, two linearments corresponded the gray hornfels and black slate member were extracted and in succession, the additional uranium potential zone was inferred. Final discrimination whether uranium-rich or not was confirmed by radiometric and uranium anomaly map. From these analysis, we finally concluded that uranium deposit originated by pyrometamorphic process was confined near the intrusive area only. On the contrary, it was found that black shale related uranium deposit is distributed and extended through out the entire survey area with south-west to north-east direction. In addition, from the linear feature analysis based on radiometric total anomaly map, the typical discontinuous characteristics were recognized in areas where uranium-contained linearments cross the faults. From the above discussion, we concluded that airborne magnetic and radiometric survey are complementary to each other. So it is preferable to carry out simultaneously for the efficient data processing and fruitful interpretation.

Handling Method of Imbalance Data for Machine Learning : Focused on Sampling (머신러닝을 위한 불균형 데이터 처리 방법 : 샘플링을 위주로)

  • Lee, Kyunam;Lim, Jongtae;Bok, Kyoungsoo;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.19 no.11
    • /
    • pp.567-577
    • /
    • 2019
  • Recently, more and more attempts have been made to solve the problems faced by academia and industry through machine learning. Accordingly, various attempts are being made to solve non-general situations through machine learning, such as deviance, fraud detection and disability detection. A variety of attempts have been made to resolve the non-normal situation in which data is distributed disproportionately, generally resulting in errors. In this paper, we propose handling method of imbalance data for machine learning. The proposed method to such problem of an imbalance in data by verifying that the population distribution of major class is well extracted. Performance Evaluations have proven the proposed method to be better than the existing methods.

Characteristics Analysis of Measurement Variables for Detecting Anomaly Signs of Thermal Runaway in Lithium-Ion Batteries (리튬이온 배터리의 열폭주 이상징후 감지를 위한 측정 변수 특성 분석)

  • LIM, BYUNG-JU;CHO, SUNG-HOON;LEE, GA-RAM;CHOI, SEOK-MIN;PARK, CHANG-DAE
    • Transactions of the Korean hydrogen and new energy society
    • /
    • v.33 no.1
    • /
    • pp.85-94
    • /
    • 2022
  • To detect anomaly signs of thermal runaway in advance, this study analyzed the signals from various sensors installed in lithium-ion batteries. The thermal runaway mechanism was analyzed, and measurement variables for anomalies of a battery cell were surface temperature, strain, and gas concentration. The changes and characteristics of three variables during the thermal runaway process were analyzed under the abuse environment: the overheat and the overcharge. In experiment, the thermal runaway of the battery proceeded in the initial developing stage, the outgassing stage, and the ignition stage. Analysis from the measured data indicated that the suitable variable to detect all stages of thermal runaway is the surface temperature of the battery, and surface strain is alternative.

Design of NePID using Anomaly Traffic Analysis and Fuzzy Cognitive Maps (비정상 트래픽 분석과 퍼지인식도를 이용한 NePID 설계)

  • Kim, Hyeock-Jin;Ryu, Sang-Ryul;Lee, Se-Yul
    • Journal of the Korea Academia-Industrial cooperation Society
    • /
    • v.10 no.4
    • /
    • pp.811-817
    • /
    • 2009
  • The rapid growth of network based IT systems has resulted in continuous research of security issues. Probe intrusion detection is an area of increasing concerns in the internet community. Recently, a number of probe intrusion detection schemes have been proposed based on various technologies. However, the techniques, which have been applied in many systems, are useful only for the existing patterns of probe intrusion. They can not detect new patterns of probe intrusion. Therefore, it is necessary to develop a new Probe Intrusion Detection technology that can find new patterns of probe intrusion. In this paper, we proposed a new network based probe intrusion detector(NePID) using anomaly traffic analysis and fuzzy cognitive maps that can detect intrusion by the denial of services attack detection method utilizing the packet analyses. The probe intrusion detection using fuzzy cognitive maps capture and analyze the packet information to detect syn flooding attack. Using the result of the analysis of decision module, which adopts the fuzzy cognitive maps, the decision module measures the degree of risk of denial of service attack and trains the response module to deal with attacks. For the performance evaluation, the "IDS Evaluation Data Set" created by MIT was used. From the simulation we obtained the max-average true positive rate of 97.094% and the max-average false negative rate of 2.936%. The true positive error rate of the NePID is similar to that of Bernhard's true positive error rate.

Performance Evaluation of Snow Detection Using Himawari-8 AHI Data (Himawari-8 AHI 적설 탐지의 성능 평가)

  • Jin, Donghyun;Lee, Kyeong-sang;Seo, Minji;Choi, Sungwon;Seong, Noh-hun;Lee, Eunkyung;Han, Hyeon-gyeong;Han, Kyung-soo
    • Korean Journal of Remote Sensing
    • /
    • v.34 no.6_1
    • /
    • pp.1025-1032
    • /
    • 2018
  • Snow Cover is a form of precipitation that is defined by snow on the surface and is the single largest component of the cryosphere that plays an important role in maintaining the energy balance between the earth's surface and the atmosphere. It affects the regulation of the Earth's surface temperature. However, since snow cover is mainly distributed in area where human access is difficult, snow cover detection using satellites is actively performed, and snow cover detection in forest area is an important process as well as distinguishing between cloud and snow. In this study, we applied the Normalized Difference Snow Index (NDSI) and the Normalized Difference Vegetation Index (NDVI) to the geostationary satellites for the snow detection of forest area in existing polar orbit satellites. On the rest of the forest area, the snow cover detection using $R_{1.61{\mu}m}$ anomaly technique and NDSI was performed. As a result of the indirect validation using the snow cover data and the Visible Infrared Imaging Radiometer (VIIRS) snow cover data, the probability of detection (POD) was 99.95 % and the False Alarm Ratio (FAR) was 16.63 %. We also performed qualitative validation using the Himawari-8 Advanced Himawari Imager (AHI) RGB image. The result showed that the areas detected by the VIIRS Snow Cover miss pixel are mixed with the area detected by the research false pixel.

Anomaly Detection in Livestock Environmental Time Series Data Using LSTM Autoencoders: A Comparison of Performance Based on Threshold Settings (LSTM 오토인코더를 활용한 축산 환경 시계열 데이터의 이상치 탐지: 경계값 설정에 따른 성능 비교)

  • Se Yeon Chung;Sang Cheol Kim
    • Smart Media Journal
    • /
    • v.13 no.4
    • /
    • pp.48-56
    • /
    • 2024
  • In the livestock industry, detecting environmental outliers and predicting data are crucial tasks. Outliers in livestock environment data, typically gathered through time-series methods, can signal rapid changes in the environment and potential unexpected epidemics. Prompt detection and response to these outliers are essential to minimize stress in livestock and reduce economic losses for farmers by early detection of epidemic conditions. This study employs two methods to experiment and compare performances in setting thresholds that define outliers in livestock environment data outlier detection. The first method is an outlier detection using Mean Squared Error (MSE), and the second is an outlier detection using a Dynamic Threshold, which analyzes variability against the average value of previous data to identify outliers. The MSE-based method demonstrated a 94.98% accuracy rate, while the Dynamic Threshold method, which uses standard deviation, showed superior performance with 99.66% accuracy.

Anomaly Diagnosis of Rotational Machinery Using Time-Series Vibration Data Based on Time-Distributed CNN-LSTM (시분할 CNN-LSTM 기반의 시계열 진동 데이터를 이용한 회전체 기계 설비의 이상 진단)

  • Kim, Min-Ki
    • Journal of Korea Multimedia Society
    • /
    • v.25 no.11
    • /
    • pp.1547-1556
    • /
    • 2022
  • As mechanical facilities are interacting with each other, the failure of some equipment can affect the entire system, so it is necessary to quickly detect and diagnose the abnormality of mechanical equipment. This study proposes a deep learning model that can effectively diagnose abnormalities in rotating machinery and equipment. CNN is widely used for feature extraction and LSTMs are known to be effective in learning sequential information. In LSTM, the number of parameters and learning time increase as the length of input data increases. In this study, we propose a method of segmenting an input segment signal into shorter-length sub-segment signals, sequentially inputting them to CNN through a time-distributed method for extracting features, and inputting them into LSTM. A failure diagnosis test was performed using the vibration data collected from the motor for ventilation equipment installed at the urban railway station. The experiment showed an accuracy of 99.784% in fault diagnosis. It shows that the proposed method is effective in the fault diagnosis of rotating machinery and equipment.