• Journal of Intelligence and Information Systems
• /
• v.27 no.3
• /
• pp.57-73
• /
• 2021

Maintenance and prevention of failure through anomaly detection of ICT infrastructure is becoming important. System monitoring data is multidimensional time series data. When we deal with multidimensional time series data, we have difficulty in considering both characteristics of multidimensional data and characteristics of time series data. When dealing with multidimensional data, correlation between variables should be considered. Existing methods such as probability and linear base, distance base, etc. are degraded due to limitations called the curse of dimensions. In addition, time series data is preprocessed by applying sliding window technique and time series decomposition for self-correlation analysis. These techniques are the cause of increasing the dimension of data, so it is necessary to supplement them. The anomaly detection field is an old research field, and statistical methods and regression analysis were used in the early days. Currently, there are active studies to apply machine learning and artificial neural network technology to this field. Statistically based methods are difficult to apply when data is non-homogeneous, and do not detect local outliers well. The regression analysis method compares the predictive value and the actual value after learning the regression formula based on the parametric statistics and it detects abnormality. Anomaly detection using regression analysis has the disadvantage that the performance is lowered when the model is not solid and the noise or outliers of the data are included. There is a restriction that learning data with noise or outliers should be used. The autoencoder using artificial neural networks is learned to output as similar as possible to input data. It has many advantages compared to existing probability and linear model, cluster analysis, and map learning. It can be applied to data that does not satisfy probability distribution or linear assumption. In addition, it is possible to learn non-mapping without label data for teaching. However, there is a limitation of local outlier identification of multidimensional data in anomaly detection, and there is a problem that the dimension of data is greatly increased due to the characteristics of time series data. In this study, we propose a CMAE (Conditional Multimodal Autoencoder) that enhances the performance of anomaly detection by considering local outliers and time series characteristics. First, we applied Multimodal Autoencoder (MAE) to improve the limitations of local outlier identification of multidimensional data. Multimodals are commonly used to learn different types of inputs, such as voice and image. The different modal shares the bottleneck effect of Autoencoder and it learns correlation. In addition, CAE (Conditional Autoencoder) was used to learn the characteristics of time series data effectively without increasing the dimension of data. In general, conditional input mainly uses category variables, but in this study, time was used as a condition to learn periodicity. The CMAE model proposed in this paper was verified by comparing with the Unimodal Autoencoder (UAE) and Multi-modal Autoencoder (MAE). The restoration performance of Autoencoder for 41 variables was confirmed in the proposed model and the comparison model. The restoration performance is different by variables, and the restoration is normally well operated because the loss value is small for Memory, Disk, and Network modals in all three Autoencoder models. The process modal did not show a significant difference in all three models, and the CPU modal showed excellent performance in CMAE. ROC curve was prepared for the evaluation of anomaly detection performance in the proposed model and the comparison model, and AUC, accuracy, precision, recall, and F1-score were compared. In all indicators, the performance was shown in the order of CMAE, MAE, and AE. Especially, the reproduction rate was 0.9828 for CMAE, which can be confirmed to detect almost most of the abnormalities. The accuracy of the model was also improved and 87.12%, and the F1-score was 0.8883, which is considered to be suitable for anomaly detection. In practical aspect, the proposed model has an additional advantage in addition to performance improvement. The use of techniques such as time series decomposition and sliding windows has the disadvantage of managing unnecessary procedures; and their dimensional increase can cause a decrease in the computational speed in inference.The proposed model has characteristics that are easy to apply to practical tasks such as inference speed and model management.

• Journal of Intelligence and Information Systems
• /
• v.26 no.4
• /
• pp.127-148
• /
• 2020

The data center is a physical environment facility for accommodating computer systems and related components, and is an essential foundation technology for next-generation core industries such as big data, smart factories, wearables, and smart homes. In particular, with the growth of cloud computing, the proportional expansion of the data center infrastructure is inevitable. Monitoring the health of these data center facilities is a way to maintain and manage the system and prevent failure. If a failure occurs in some elements of the facility, it may affect not only the relevant equipment but also other connected equipment, and may cause enormous damage. In particular, IT facilities are irregular due to interdependence and it is difficult to know the cause. In the previous study predicting failure in data center, failure was predicted by looking at a single server as a single state without assuming that the devices were mixed. Therefore, in this study, data center failures were classified into failures occurring inside the server (Outage A) and failures occurring outside the server (Outage B), and focused on analyzing complex failures occurring within the server. Server external failures include power, cooling, user errors, etc. Since such failures can be prevented in the early stages of data center facility construction, various solutions are being developed. On the other hand, the cause of the failure occurring in the server is difficult to determine, and adequate prevention has not yet been achieved. In particular, this is the reason why server failures do not occur singularly, cause other server failures, or receive something that causes failures from other servers. In other words, while the existing studies assumed that it was a single server that did not affect the servers and analyzed the failure, in this study, the failure occurred on the assumption that it had an effect between servers. In order to define the complex failure situation in the data center, failure history data for each equipment existing in the data center was used. There are four major failures considered in this study: Network Node Down, Server Down, Windows Activation Services Down, and Database Management System Service Down. The failures that occur for each device are sorted in chronological order, and when a failure occurs in a specific equipment, if a failure occurs in a specific equipment within 5 minutes from the time of occurrence, it is defined that the failure occurs simultaneously. After configuring the sequence for the devices that have failed at the same time, 5 devices that frequently occur simultaneously within the configured sequence were selected, and the case where the selected devices failed at the same time was confirmed through visualization. Since the server resource information collected for failure analysis is in units of time series and has flow, we used Long Short-term Memory (LSTM), a deep learning algorithm that can predict the next state through the previous state. In addition, unlike a single server, the Hierarchical Attention Network deep learning model structure was used in consideration of the fact that the level of multiple failures for each server is different. This algorithm is a method of increasing the prediction accuracy by giving weight to the server as the impact on the failure increases. The study began with defining the type of failure and selecting the analysis target. In the first experiment, the same collected data was assumed as a single server state and a multiple server state, and compared and analyzed. The second experiment improved the prediction accuracy in the case of a complex server by optimizing each server threshold. In the first experiment, which assumed each of a single server and multiple servers, in the case of a single server, it was predicted that three of the five servers did not have a failure even though the actual failure occurred. However, assuming multiple servers, all five servers were predicted to have failed. As a result of the experiment, the hypothesis that there is an effect between servers is proven. As a result of this study, it was confirmed that the prediction performance was superior when the multiple servers were assumed than when the single server was assumed. In particular, applying the Hierarchical Attention Network algorithm, assuming that the effects of each server will be different, played a role in improving the analysis effect. In addition, by applying a different threshold for each server, the prediction accuracy could be improved. This study showed that failures that are difficult to determine the cause can be predicted through historical data, and a model that can predict failures occurring in servers in data centers is presented. It is expected that the occurrence of disability can be prevented in advance using the results of this study.

• Geophysics and Geophysical Exploration
• /
• v.19 no.1
• /
• pp.1-10
• /
• 2016

The hypocenter distribution of microseismic events generated by hydraulic fracturing for shale gas development provides essential information for understanding characteristics of fracture network. In this study, we evaluate how inaccurate velocity models influence the inversion results of two widely used location programs, hypoellipse and hypoDD, which are developed based on an iterative linear inversion. We assume that 98 stations are densely located inside the circle with a radius of 4 km and 5 artificial hypocenter sets (S0 ~ S4) are located from the center of the network to the south with 1 km interval. Each hypocenter set contains 25 events placed on the plane. To quantify accuracies of the inversion results, we defined 6 parameters: difference between average hypocenters of assumed and inverted locations, $d_1$; ratio of assumed and inverted areas estimated by hypocenters, r; difference between dip of the reference plane and the best fitting plane for determined hypocenters, ${\theta}$; difference between strike of the reference plane and the best fitting plane for determined hypocenters, ${\phi}$; root-mean-square distance between hypocenters and the best fitting plane, $d_2$; root-mean-square error in horizontal direction on the best fitting plane, $d_3$. Synthetic travel times are calculated for the reference model having 1D layered structure and the inaccurate velocity model for the inversion is constructed by using normal distribution with standard deviations of 0.1, 0.2, and 0.3 km/s, respectively, with respect to the reference model. The parameters $d_1$, r, ${\theta}$, and $d_2$ show positive correlation with the level of velocity perturbations, but the others are not sensitive to the perturbations except S4, which is located at the outer boundary of the network. In cases of S0, S1, S2, and S3, hypoellipse and hypoDD provide similar results for $d_1$. However, for other parameters, hypoDD shows much better results and errors of locations can be reduced by about several meters regardless of the level of perturbations. In light of the purpose to understand the characteristics of hydraulic fracturing, $1{\sigma}$ error of velocity structure should be under 0.2 km/s in hypoellipse and 0.3 km/s in hypoDD.

• Journal of Cadastre & Land InformatiX
• /
• v.48 no.2
• /
• pp.79-98
• /
• 2018

The purpose of this study is to suggest a model analysing spatio-temporal characteristics of the civil complaints for the officially assessed land price based on big data mining. Specifically, in this study, the underlying reasons for the civil complaints were found from the spatio-temporal perspectives, rather than the institutional factors, and a model was suggested monitoring a trend of the occurrence of such complaints. The official documents of 6,481 civil complaints for the officially assessed land price in the district of Jung-gu of Incheon Metropolitan City over the period from 2006 to 2015 along with their temporal and spatial poperties were collected and used for the analysis. Frequencies of major key words were examined by using a text mining method. Correlations among mafor key words were studied through the social network analysis. By calculating term frequency(TF) and term frequency-inverse document frequency(TF-IDF), which correspond to the weighted value of key words, I identified the major key words for the occurrence of the civil complaint for the officially assessed land price. Then the spatio-temporal characteristics of the civil complaints were examined by analysing hot spot based on the statistics of Getis-Ord $Gi^*$. It was found that the characteristic of civil complaints for the officially assessed land price were changing, forming a cluster that is linked spatio-temporally. Using text mining and social network analysis method, we could find out that the occurrence reason of civil complaints for the officially assessed land price could be identified quantitatively based on natural language. TF and TF-IDF, the weighted averages of key words, can be used as main explanatory variables to analyze spatio-temporal characteristics of civil complaints for the officially assessed land price since these statistics are different over time across different regions.

• Korean Journal of Agricultural and Forest Meteorology
• /
• v.21 no.1
• /
• pp.1-28
• /
• 2019

This study analyzed the density and mortality rate of Korean fir at 9 sites where individuals of Korean firs were marked into the live and dead trees with coordinates on orthorectified aerial images by digital photogrammetric system. As a result of the analysis, Korean fir in each site showed considerable heterogeneity in density and mortality rate depending on the location within site. This make it possible to assume that death of Korean fir can occur by specific factors that vary depending on the location. Based on the analyzed densities and mortality rates of Korea fir, we investigated the correlation between topographic factors such as altitude, terrain slope, drainage network, solar radiation, aspect and the death of Korean fir. The density of Korean fir increases with altitude, and the mortality rate also increases. A negative correlation is found between the terrain slope and the mortality rate, and the mortality rate is higher in the gentle slope where the drainage network is less developed. In addition, it is recognized that depending on the aspect, the mortality rate varies greatly, and the mean solar radiation is higher in live Korean fir-dominant area than in dead Korean fir-dominant area. Overall, the mortality rate of Korean fir in Mt. Halla area is relatively higher in areas with relatively low terrain slope and low solar radiation. Considering the results of previous studies that the terrain slope has a strong negative correlation with soil moisture and the relationship between solar radiation and evaporation, these results lead us to infer that excess soil moisture is the cause of Korean fir mortality. These inferences are supported by a series of climate change phenomena such as precipitation increase, evaporation decrease, and reduced sunshine duration in the Korean peninsula including Jeju Island, increase in mortality rate along with increased precipitation according to the elevation of Mt. Halla and the vegetation change in the mountain. It is expected that the spatial patterns in the density and mortality rate of Korean fir, which are controlled by topography such as altitude, slope, aspect, solar radiation, drainage network, can be used as spatial variables in future numerical modeling studies on the death or decline of Korean fir. In addition, the method of forest distribution survey using the orthorectified aerial images can be widely used as a numerical monitoring technique in long - term vegetation change research.

• Journal of Environmental Impact Assessment
• /
• v.32 no.2
• /
• pp.134-145
• /
• 2023

Currently, the floodplains of major rivers are transforming into various types of waterfront spaces according to the increase in leisure activities and improved accessibility. In general, waterfront activities in river channels tend to be concentrated in summer, and the waterfront activities during this period directly affect water quality. Accordingly, it is necessary to accurately compare and evaluate the characteristics and water quality of waterfront activities during the period when waterfront activities are concentrated. In this study, the following research was conducted to compare and analyze the current status of waterfront activities of users of waterfront areas and the water quality of waterfront areas. First, three waterfront areas were selected for investigation using the information from the Ministry of Environment's water quality measurement network. Second, a survey was conducted on the satisfaction and types of waterfront activities targeting users of waterfront areas. Third, water quality grades were calculated based on monthly water quality measurement factors and compared. Fourth, statistical analysis (one-way analysis of variance) was conducted to see if there was a significant difference in water quality characteristics between periods of high waterfront activity and periods of low waterfront activity using water quality measurement data for the last 5 years. As a result of this analysis, the following conclusions were drawn in this study. First, the use of waterfront activities was investigated in the order of camping, water skiing, fishing, swimming, and rafting. Second, satisfaction factors for waterfront activities were investigated in the order of activity convenience, water quality, waterlandscape, transportation access convenience, and temperature. Third, it was found that satisfaction with water quality in waterfront areas was generally unsatisfactory regardless of the water quality grade presented by the competent authority. Fourth, as a result of comparing the water quality measurement network data of the Ministry of Environment by water quality grade, generally good grades were found, and in particular, there was a difference in grade frequency by season in the BOD category. Fifth, as a result of statistical analysis (one-way ANOVA) of water quality monitoring network data by season, there were statistically significant differences in COD, BOD, TP, and TOC except for DO. Considering the results of these studies, it is judged that it is necessary to prepare a comprehensive management system for water quality improvement in the waterfront zone and to improve water quality during periods of high waterfront activity, and to prepare a water quality forecasting system for waterfront areas in the future.

• Korean Journal of Remote Sensing
• /
• v.39 no.6_1
• /
• pp.1413-1425
• /
• 2023

The increasing frequency of wildfires due to climate change is causing extreme loss of life and property. They cause loss of vegetation and affect ecosystem changes depending on their intensity and occurrence. Ecosystem changes, in turn, affect wildfire occurrence, causing secondary damage. Thus, accurate estimation of the areas affected by wildfires is fundamental. Satellite remote sensing is used for forest fire detection because it can rapidly acquire topographic and meteorological information about the affected area after forest fires. In addition, deep learning algorithms such as convolutional neural networks (CNN) and transformer models show high performance for more accurate monitoring of fire-burnt regions. To date, the application of deep learning models has been limited, and there is a scarcity of reports providing quantitative performance evaluations for practical field utilization. Hence, this study emphasizes a comparative analysis, exploring performance enhancements achieved through both model selection and data design. This study examined deep learning models for detecting wildfire-damaged areas using Landsat 8 satellite images in California. Also, we conducted a comprehensive comparison and analysis of the detection performance of multiple models, such as U-Net and High-Resolution Network-Object Contextual Representation (HRNet-OCR). Wildfire-related spectral indices such as normalized difference vegetation index (NDVI) and normalized burn ratio (NBR) were used as input channels for the deep learning models to reflect the degree of vegetation cover and surface moisture content. As a result, the mean intersection over union (mIoU) was 0.831 for U-Net and 0.848 for HRNet-OCR, showing high segmentation performance. The inclusion of spectral indices alongside the base wavelength bands resulted in increased metric values for all combinations, affirming that the augmentation of input data with spectral indices contributes to the refinement of pixels. This study can be applied to other satellite images to build a recovery strategy for fire-burnt areas.

• Korean Journal of Remote Sensing
• /
• v.39 no.5_3
• /
• pp.979-995
• /
• 2023

As wildfires are difficult to predict, real-time monitoring is crucial for a timely response. Geostationary satellite images are very useful for active fire detection because they can monitor a vast area with high temporal resolution (e.g., 2 min). Existing satellite-based active fire detection algorithms detect thermal outliers using threshold values based on the statistical analysis of brightness temperature. However, the difficulty in establishing suitable thresholds for such threshold-based methods hinders their ability to detect fires with low intensity and achieve generalized performance. In light of these challenges, machine learning has emerged as a potential-solution. Until now, relatively simple techniques such as random forest, Vanilla convolutional neural network (CNN), and U-net have been applied for active fire detection. Therefore, this study proposed an active fire detection algorithm using state-of-the-art (SOTA) deep learning techniques using data from the Advanced Himawari Imager and evaluated it over East Asia and Australia. The SOTA model was developed by applying EfficientNet and lion optimizer, and the results were compared with the model using the Vanilla CNN structure. EfficientNet outperformed CNN with F1-scores of 0.88 and 0.83 in East Asia and Australia, respectively. The performance was better after using weighted loss, equal sampling, and image augmentation techniques to fix data imbalance issues compared to before the techniques were used, resulting in F1-scores of 0.92 in East Asia and 0.84 in Australia. It is anticipated that timely responses facilitated by the SOTA deep learning-based approach for active fire detection will effectively mitigate the damage caused by wildfires.

• Journal of Intelligence and Information Systems
• /
• v.26 no.2
• /
• pp.131-145
• /
• 2020

In line with the trend of industrial innovation, IoT technology utilized in a variety of fields is emerging as a key element in creation of new business models and the provision of user-friendly services through the combination of big data. The accumulated data from devices with the Internet-of-Things (IoT) is being used in many ways to build a convenience-based smart system as it can provide customized intelligent systems through user environment and pattern analysis. Recently, it has been applied to innovation in the public domain and has been using it for smart city and smart transportation, such as solving traffic and crime problems using CCTV. In particular, it is necessary to comprehensively consider the easiness of securing real-time service data and the stability of security when planning underground services or establishing movement amount control information system to enhance citizens' or commuters' convenience in circumstances with the congestion of public transportation such as subways, urban railways, etc. However, previous studies that utilize image data have limitations in reducing the performance of object detection under private issue and abnormal conditions. The IoT device-based sensor data used in this study is free from private issue because it does not require identification for individuals, and can be effectively utilized to build intelligent public services for unspecified people. Especially, sensor data stored by the IoT device need not be identified to an individual, and can be effectively utilized for constructing intelligent public services for many and unspecified people as data free form private issue. We utilize the IoT-based infrared sensor devices for an intelligent pedestrian tracking system in metro service which many people use on a daily basis and temperature data measured by sensors are therein transmitted in real time. The experimental environment for collecting data detected in real time from sensors was established for the equally-spaced midpoints of 4×4 upper parts in the ceiling of subway entrances where the actual movement amount of passengers is high, and it measured the temperature change for objects entering and leaving the detection spots. The measured data have gone through a preprocessing in which the reference values for 16 different areas are set and the difference values between the temperatures in 16 distinct areas and their reference values per unit of time are calculated. This corresponds to the methodology that maximizes movement within the detection area. In addition, the size of the data was increased by 10 times in order to more sensitively reflect the difference in temperature by area. For example, if the temperature data collected from the sensor at a given time were 28.5℃, the data analysis was conducted by changing the value to 285. As above, the data collected from sensors have the characteristics of time series data and image data with 4×4 resolution. Reflecting the characteristics of the measured, preprocessed data, we finally propose a hybrid algorithm that combines CNN in superior performance for image classification and LSTM, especially suitable for analyzing time series data, as referred to CNN-LSTM (Convolutional Neural Network-Long Short Term Memory). In the study, the CNN-LSTM algorithm is used to predict the number of passing persons in one of 4×4 detection areas. We verified the validation of the proposed model by taking performance comparison with other artificial intelligence algorithms such as Multi-Layer Perceptron (MLP), Long Short Term Memory (LSTM) and RNN-LSTM (Recurrent Neural Network-Long Short Term Memory). As a result of the experiment, proposed CNN-LSTM hybrid model compared to MLP, LSTM and RNN-LSTM has the best predictive performance. By utilizing the proposed devices and models, it is expected various metro services will be provided with no illegal issue about the personal information such as real-time monitoring of public transport facilities and emergency situation response services on the basis of congestion. However, the data have been collected by selecting one side of the entrances as the subject of analysis, and the data collected for a short period of time have been applied to the prediction. There exists the limitation that the verification of application in other environments needs to be carried out. In the future, it is expected that more reliability will be provided for the proposed model if experimental data is sufficiently collected in various environments or if learning data is further configured by measuring data in other sensors.

• Korean Journal of Soil Science and Fertilizer
• /
• v.43 no.3
• /
• pp.374-383
• /
• 2010

Microwave radar can penetrate cloud cover regardless of weather conditions and can be used day and night. Especially a ground-based polarimetric scatterometer has advantages of monitoring crop conditions continuously with full polarization and different frequencies. Kim et al. (2009) have measured backscattering coefficients of paddy rice using L-, C-, X-band scatterometer system with full polarization and various angles during the rice growth period and have revealed the necessity of near-continuous automatic measurement to eliminate the difficulties, inaccuracy and sparseness of data acquisitions arising from manual operation of the system. In this study, we constructed an X-band automatic scatterometer system, analyzed scattering characteristics of paddy rice from X-band scatterometer data and estimated rice growth parameter using backscattering coefficients in X-band. The system was installed inside a shelter in an experimental paddy field at the National Academy of Agricultural Science (NAAS) before rice transplanting. The scatterometer system consists of X-band antennas, HP8720D vector network analyzer, RF cables and personal computer that controls frequency, polarization and data storage. This system using automatically measures fully-polarimetric backscattering coefficients of rice crop every 10 minutes. The backscattering coefficients were calculated from the measured data at a fixed incidence angle of $45^{\circ}$ and with full polarization (HH, VV, HV, VH) by applying the radar equation and compared with rice growth data such as plant height, stem number, fresh dry weight and Leaf Area Index (LAI) that were collected at the same time of each rice growth parameter. We examined the temporal behaviour of the backscattering coefficients of the rice crop at X-band during rice growth period. The HH-, VV-polarization backscattering coefficients steadily increased toward panicle initiation stage, thereafter decreased and again increased in early-September. We analyzed the relationships between backscattering coefficients in X-band and plant parameters and predicted the rice growth parameters using backscattering coefficients. It was confirmed that X-band is sensitive to grain maturity at near harvesting season.