• Journal of the Korea Society of Computer and Information
• /
• v.29 no.4
• /
• pp.39-46
• /
• 2024

Many researchers make efforts to evaluate water quality using various models. Such models require a dataset without missing values, but in real world, most datasets include missing values for various reasons. Simple deletion of samples having missing value(s) could distort distribution of the underlying data and pose a significant risk of biasing the model's inference when the missing mechanism is not MCAR. In this study, to explore the most appropriate technique for handing missing values in water quality data, several imputation techniques were experimented based on existing KNN and MICE imputation with/without the generative neural network model, Autoencoder(AE) and Denoising Autoencoder(DAE). The results shows that KNN and MICE combined imputation without generative networks provides the closest estimated values to the true values. When evaluating binary classification models based on support vector machine and ensemble algorithms after applying the combined imputation technique to the observed water quality dataset with missing values, it shows better performance in terms of Accuracy, F1 score, RoC-AuC score and MCC compared to those evaluated after deleting samples having missing values.

• International Journal of Internet, Broadcasting and Communication
• /
• v.13 no.2
• /
• pp.7-13
• /
• 2021

Machine vision is a technology that helps the computer as if a person recognizes and determines things. In recent years, as advanced technologies such as optical systems, artificial intelligence and big data advanced in conventional machine vision system became more accurate quality inspection and it increases the manufacturing efficiency. In machine vision systems using deep learning, the image quality of the input image is very important. However, most images obtained in the industrial field for quality inspection typically contain noise. This noise is a major factor in the performance of the machine vision system. Therefore, in order to improve the performance of the machine vision system, it is necessary to eliminate the noise of the image. There are lots of research being done to remove noise from the image. In this paper, we propose an autoencoder based machine vision system to eliminate noise in the image. Through experiment proposed model showed better performance compared to the basic autoencoder model in denoising and image reconstruction capability for MNIST and fashion MNIST data sets.

• Journal of Korea Multimedia Society
• /
• v.23 no.8
• /
• pp.965-976
• /
• 2020

We present in this work a segmentation method of E. coli bacterial images generated via phase contrast microscopy using a deep learning based hybrid feature generation. Unlike conventional machine learning methods that use the hand-crafted features, we adopt the denoising autoencoder in order to generate a precise and accurate representation of the pixels. We first construct a hybrid vector that combines original image, difference of Gaussians and image gradients. The created hybrid features are then given to a deep autoencoder that learns the pixels' internal dependencies and the cells' shape and boundary information. The latent representations learned by the autoencoder are used as the inputs of a softmax classification layer and the direct outputs from the classifier represent the coarse segmentation mask. Finally, the classifier's outputs are used as prior information for a graph partitioning based fine segmentation. We demonstrate that the proposed hybrid vector representation manages to preserve the global shape and boundary information of the cells, allowing to retrieve the majority of the cellular patterns without the need of any post-processing.

• Journal of the Korea Convergence Society
• /
• v.10 no.5
• /
• pp.17-26
• /
• 2019

We aimed to build a patient-based allergy prevention system using the smartphone and focused on the region of interest (ROI) extraction method for Optical Character Recognition (OCR) in the general environment. However, the current ROI extraction method has shown good performance in the experimental environment, but the performance in the real environment was not good due to the noisy background. Therefore, in this paper, we propose the compared methods of reducing noisy background to solve the ROI extraction problem. There five methods used as a SMF, DIN, Denoising Autoencoder(DAE), DAE with Convolution Neural Network(DAECNN) and median filter(MF) with DAECNN (MF+DAECNN). We have shown that our proposed DAECNN and MF+DAECNN methods are 69%, respectively, which is relatively higher than the conventional DAE method 55%. The verification of performance improvement uses MSE, PSNR and SSIM. The system has implemented OpenCV, C++ and Python, including its performance, is tested on real images.

• KIPS Transactions on Software and Data Engineering
• /
• v.12 no.8
• /
• pp.355-364
• /
• 2023

As advanced cyber threats continue to increase in recent years, it is difficult to detect new types of cyber attacks with existing pattern or signature-based intrusion detection method. Therefore, research on anomaly detection methods using data learning-based artificial intelligence technology is increasing. In addition, supervised learning-based anomaly detection methods are difficult to use in real environments because they require sufficient labeled data for learning. Research on an unsupervised learning-based method that learns from normal data and detects an anomaly by finding a pattern in the data itself has been actively conducted. Therefore, this study aims to extract a latent vector that preserves useful sequence information from sequence log data and develop an anomaly detection learning model using the extracted latent vector. Word2Vec was used to create a dense vector representation corresponding to the characteristics of each sequence, and an unsupervised autoencoder was developed to extract latent vectors from sequence data expressed as dense vectors. The developed autoencoder model is a recurrent neural network GRU (Gated Recurrent Unit) based denoising autoencoder suitable for sequence data, a one-dimensional convolutional neural network-based autoencoder to solve the limited short-term memory problem that GRU can have, and an autoencoder combining GRU and one-dimensional convolution was used. The data used in the experiment is time-series-based NGIDS (Next Generation IDS Dataset) data, and as a result of the experiment, an autoencoder that combines GRU and one-dimensional convolution is better than a model using a GRU-based autoencoder or a one-dimensional convolution-based autoencoder. It was efficient in terms of learning time for extracting useful latent patterns from training data, and showed stable performance with smaller fluctuations in anomaly detection performance.

• Nuclear Engineering and Technology
• /
• v.53 no.3
• /
• pp.888-893
• /
• 2021

The International Atomic Energy Agency has developed a tomographic imaging system for accomplishing the total fuel rod-by-rod verification time of fuel assemblies within the order of 1-2 h, however, there are still limitations for some fuel types. The aim of this study is to develop a deep learning-based denoising process resulting in increasing the tomographic image acquisition speed of fuel assembly compared to the conventional techniques. Convolutional AutoEncoder (CAE) was employed for denoising the low-quality images reconstructed by filtered back-projection (FBP) algorithm. The image data set was constructed by the Monte Carlo method with the FBP and ground truth (GT) images for 511 patterns of missing fuel rods. The de-noising performance of the CAE model was evaluated by comparing the pixel-by-pixel subtracted images between the GT and FBP images and the GT and CAE images; the average differences of the pixel values for the sample image 1, 2, and 3 were 7.7%, 28.0% and 44.7% for the FBP images, and 0.5%, 1.4% and 1.9% for the predicted image, respectively. Even for the FBP images not discriminable the source patterns, the CAE model could successfully estimate the patterns similarly with the GT image.

• Journal of IKEEE
• /
• v.24 no.2
• /
• pp.632-637
• /
• 2020

The non-intrusive load monitoring (NILM) is an important way to cost-effective real-time monitoring the energy consumption and time of use for each appliance in a home or business using aggregated energy from a single recording meter. In this paper, we collect from the smart farm's power consumption data acquisition system to the server via an LTE modem, converted the total power consumption, and the power of individual electric devices into HDF5 format and performed NILM analysis. We perform NILM analysis using open source denoising autoencoder (DAE), long short-term memory (LSTM), gated recurrent unit (GRU), and sequence-to-point (seq2point) learning methods.

• Economic and Environmental Geology
• /
• v.54 no.1
• /
• pp.77-89
• /
• 2021

In this study, a method has been proposed to improve the performance of hydraulic property estimation model developed by Jeong et al. (2020). In their study, low-dimensional features of the annual groundwater level (GWL) fluctuation patterns extracted based on a Denoising autoencoder (DAE) was used to develop a regression model for predicting hydraulic properties of an aquifer. However, low-dimensional features of the DAE are highly dependent on the precipitation pattern even if the GWL is monitored at the same location, causing uncertainty in hydraulic property estimation of the regression model. To solve the above problem, a process for generating the GWL fluctuation pattern for conditioning the precipitation is proposed based on a conditional variational autoencoder (CVAE). The CVAE trains a statistical relationship between GWL fluctuation and precipitation pattern. The actual GWL and precipitation data monitored on a total of 71 monitoring stations over 10 years in South Korea was applied to validate the effect of using CVAE. As a result, the trained CVAE model reasonably generated GWL fluctuation pattern with the conditioning of various precipitation patterns for all the monitoring locations. Based on the trained CVAE model, the low-dimensional features of the GWL fluctuation pattern without interference of different precipitation patterns were extracted for all monitoring stations, and they were compared to the features extracted based on the DAE. Consequently, it can be confirmed that the statistical consistency of the features extracted using CVAE is improved compared to DAE. Thus, we conclude that the proposed method may be useful in extracting a more accurate feature of GWL fluctuation pattern affected solely by hydraulic characteristics of the aquifer, which would be followed by the improved performance of the previously developed regression model.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2022.11a
• /
• pp.52-54
• /
• 2022

Aerial images are being one of the important aspects of satellite imagery, delivers effective information on landcovers. Their special characteristics includes the viewpoint from space which clarifies data related to land examining processes. Aerial images taken by satellites employed radio waves to wirelessly transmit images to ground stations. Due to transmission errors, images get distorted and unable to perform in landcover examining. This paper proposes an aerial image enhancement method using deep autoencoders. A properly trained autoencoder can enhance an aerial image to a considerable level of improvement. Results showed that the achieved enhancement is better than that was obtained from traditional image denoising methods.

• Journal of the Korea Society of Computer and Information
• /
• v.28 no.3
• /
• pp.25-33
• /
• 2023

Noise generated during the acquisition and transmission of CT images acts as a factor that degrades image quality. Therefore, noise removal to solve this problem is an important preprocessing process in image processing. In this paper, we remove noise by using a deformable convolutional autoencoder (DeCAE) model in which deformable convolution operation is applied instead of the existing convolution operation in the convolutional autoencoder (CAE) model of deep learning. Here, the deformable convolution operation can extract features of an image in a more flexible area than the conventional convolution operation. The proposed DeCAE model has the same encoder-decoder structure as the existing CAE model, but the encoder is composed of deformable convolutional layers and the decoder is composed of conventional convolutional layers for efficient noise removal. To evaluate the performance of the DeCAE model proposed in this paper, experiments were conducted on CT images corrupted by various noises, that is, Gaussian noise, impulse noise, and Poisson noise. As a result of the performance experiment, the DeCAE model has more qualitative and quantitative measures than the traditional filters, that is, the Mean filter, Median filter, Bilateral filter and NL-means method, as well as the existing CAE models, that is, MAE (Mean Absolute Error), PSNR (Peak Signal-to-Noise Ratio) and SSIM. (Structural Similarity Index Measure) showed excellent results.