• Title/Summary/Keyword: Input preprocessing

Search Result 296, Processing Time 0.022 seconds

A Study on the Applicability of Machine Learning Algorithms for Detecting Hydraulic Outliers in a Borehole (시추공 수리 이상점 탐지를 위한 기계학습 알고리즘의 적용성 연구)

  • Seungbeom Choi; Kyung-Woo Park;Changsoo Lee
    • Tunnel and Underground Space
    • /
    • v.33 no.6
    • /
    • pp.561-573
    • /
    • 2023
  • Korea Atomic Energy Research Institute (KAERI) constructed the KURT (KAERI Underground Research Tunnel) to analyze the hydrogeological/geochemical characteristics of deep rock mass. Numerous boreholes have been drilled to conduct various field tests. The selection of suitable investigation intervals within a borehole is of great importance. When objectives are centered around hydraulic flow and groundwater sampling, intervals with sufficient groundwater flow are the most suitable. This study defines such points as hydraulic outliers and aimed to detect them using borehole geophysical logging data (temperature and EC) from a 1 km depth borehole. For systematic and efficient outlier detection, machine learning algorithms, such as DBSCAN, OCSVM, kNN, and isolation forest, were applied and their applicability was assessed. Following data preprocessing and algorithm optimization, the four algorithms detected 55, 12, 52, and 68 outliers, respectively. Though this study confirms applicability of the machine learning algorithms, it is suggested that further verification and supplements are desirable since the input data were relatively limited.

Time Series Analysis for Predicting Deformation of Earth Retaining Walls (시계열 분석을 이용한 흙막이 벽체 변형 예측)

  • Seo, Seunghwan;Chung, Moonkyung
    • Journal of the Korean Geotechnical Society
    • /
    • v.40 no.2
    • /
    • pp.65-79
    • /
    • 2024
  • This study employs traditional statistical auto-regressive integrated moving average (ARIMA) and deep learning-based long short-term memory (LSTM) models to predict the deformation of earth retaining walls using inclinometer data from excavation sites. It compares the predictive capabilities of both models. The ARIMA model excels in analyzing linear patterns as time progresses, while the LSTM model is adept at handling complex nonlinear patterns and long-term dependencies in the data. This research includes preprocessing of inclinometer measurement data, performance evaluation across various data lengths and input conditions, and demonstrates that the LSTM model provides statistically significant improvements in prediction accuracy over the ARIMA model. The findings suggest that LSTM models can effectively assess the stability of retaining walls at excavation sites. Additionally, this study is expected to contribute to the development of safety monitoring systems at excavation sites and the advancement of time series prediction models.

Detection of Underwater Transient Signals Using Noise Suppression Module of EVRC Speech Codec (EVRC 음성부호화기의 잡음억제단을 이용한 수중 천이신호 검출)

  • Kim, Tae-Hwan;Bae, Keun-Sung
    • The Journal of the Acoustical Society of Korea
    • /
    • v.26 no.6
    • /
    • pp.301-305
    • /
    • 2007
  • In this paper, we propose a simple algorithm for detecting underwater transient signals on the fact that the frequency range of underwater transient signals is similar to audio frequency. For this, we use a preprocessing module of EVRC speech codec that is the standard speech codec of the mobile communications. If a signal is entered into EVRC noise suppression module, we can get some parameters such as the update flag, the energy of each channel, the noise suppressed signal, the energy of input signal, the energy of background noise, and the energy of enhanced signal. Therefore the energy of the enhanced signal that is normalized with the energy of the background noise is compared with the pre-defined detection threshold, and then we can detect the transient signal. And the detection threshold is updated using the previous value in the noisy period. The experimental result shows that the proposed algorithm has $0{\sim}4% error rate in the AWGN or the colored noise environment.

A Novel, Deep Learning-Based, Automatic Photometric Analysis Software for Breast Aesthetic Scoring

  • Joseph Kyu-hyung Park;Seungchul Baek;Chan Yeong Heo;Jae Hoon Jeong;Yujin Myung
    • Archives of Plastic Surgery
    • /
    • v.51 no.1
    • /
    • pp.30-35
    • /
    • 2024
  • Background Breast aesthetics evaluation often relies on subjective assessments, leading to the need for objective, automated tools. We developed the Seoul Breast Esthetic Scoring Tool (S-BEST), a photometric analysis software that utilizes a DenseNet-264 deep learning model to automatically evaluate breast landmarks and asymmetry indices. Methods S-BEST was trained on a dataset of frontal breast photographs annotated with 30 specific landmarks, divided into an 80-20 training-validation split. The software requires the distances of sternal notch to nipple or nipple-to-nipple as input and performs image preprocessing steps, including ratio correction and 8-bit normalization. Breast asymmetry indices and centimeter-based measurements are provided as the output. The accuracy of S-BEST was validated using a paired t-test and Bland-Altman plots, comparing its measurements to those obtained from physical examinations of 100 females diagnosed with breast cancer. Results S-BEST demonstrated high accuracy in automatic landmark localization, with most distances showing no statistically significant difference compared with physical measurements. However, the nipple to inframammary fold distance showed a significant bias, with a coefficient of determination ranging from 0.3787 to 0.4234 for the left and right sides, respectively. Conclusion S-BEST provides a fast, reliable, and automated approach for breast aesthetic evaluation based on 2D frontal photographs. While limited by its inability to capture volumetric attributes or multiple viewpoints, it serves as an accessible tool for both clinical and research applications.

Investigating Data Preprocessing Algorithms of a Deep Learning Postprocessing Model for the Improvement of Sub-Seasonal to Seasonal Climate Predictions (계절내-계절 기후예측의 딥러닝 기반 후보정을 위한 입력자료 전처리 기법 평가)

  • Uran Chung;Jinyoung Rhee;Miae Kim;Soo-Jin Sohn
    • Korean Journal of Agricultural and Forest Meteorology
    • /
    • v.25 no.2
    • /
    • pp.80-98
    • /
    • 2023
  • This study explores the effectiveness of various data preprocessing algorithms for improving subseasonal to seasonal (S2S) climate predictions from six climate forecast models and their Multi-Model Ensemble (MME) using a deep learning-based postprocessing model. A pipeline of data transformation algorithms was constructed to convert raw S2S prediction data into the training data processed with several statistical distribution. A dimensionality reduction algorithm for selecting features through rankings of correlation coefficients between the observed and the input data. The training model in the study was designed with TimeDistributed wrapper applied to all convolutional layers of U-Net: The TimeDistributed wrapper allows a U-Net convolutional layer to be directly applied to 5-dimensional time series data while maintaining the time axis of data, but every input should be at least 3D in U-Net. We found that Robust and Standard transformation algorithms are most suitable for improving S2S predictions. The dimensionality reduction based on feature selections did not significantly improve predictions of daily precipitation for six climate models and even worsened predictions of daily maximum and minimum temperatures. While deep learning-based postprocessing was also improved MME S2S precipitation predictions, it did not have a significant effect on temperature predictions, particularly for the lead time of weeks 1 and 2. Further research is needed to develop an optimal deep learning model for improving S2S temperature predictions by testing various models and parameters.

Real-Time Face Recognition Based on Subspace and LVQ Classifier (부분공간과 LVQ 분류기에 기반한 실시간 얼굴 인식)

  • Kwon, Oh-Ryun;Min, Kyong-Pil;Chun, Jun-Chul
    • Journal of Internet Computing and Services
    • /
    • v.8 no.3
    • /
    • pp.19-32
    • /
    • 2007
  • This paper present a new face recognition method based on LVQ neural net to construct a real time face recognition system. The previous researches which used PCA, LDA combined neural net usually need much time in training neural net. The supervised LVQ neural net needs much less time in training and can maximize the separability between the classes. In this paper, the proposed method transforms the input face image by PCA and LDA sequentially into low-dimension feature vectors and recognizes the face through LVQ neural net. In order to make the system robust to external light variation, light compensation is performed on the detected face by max-min normalization method as preprocessing. PCA and LDA transformations are applied to the normalized face image to produce low-level feature vectors of the image. In order to determine the initial centers of LVQ and speed up the convergency of the LVQ neural net, the K-Means clustering algorithm is adopted. Subsequently, the class representative vectors can be produced by LVQ2 training using initial center vectors. The face recognition is achieved by using the euclidean distance measure between the center vector of classes and the feature vector of input image. From the experiments, we can prove that the proposed method is more effective in the recognition ratio for the cases of still images from ORL database and sequential images rather than using conventional PCA of a hybrid method with PCA and LDA.

  • PDF

Implementation of a Spam Message Filtering System using Sentence Similarity Measurements (문장유사도 측정 기법을 통한 스팸 필터링 시스템 구현)

  • Ou, SooBin;Lee, Jongwoo
    • KIISE Transactions on Computing Practices
    • /
    • v.23 no.1
    • /
    • pp.57-64
    • /
    • 2017
  • Short message service (SMS) is one of the most important communication methods for people who use mobile phones. However, illegal advertising spam messages exploit people because they can be used without the need for friend registration. Recently, spam message filtering systems that use machine learning have been developed, but they have some disadvantages such as requiring many calculations. In this paper, we implemented a spam message filtering system using the set-based POI search algorithm and sentence similarity without servers. This algorithm can judge whether the input query is a spam message or not using only letter composition without any server computing. Therefore, we can filter the spam message although the input text message has been intentionally modified. We added a specific preprocessing option which aims to enable spam filtering. Based on the experimental results, we observe that our spam message filtering system shows better performance than the original set-based POI search algorithm. We evaluate the proposed system through extensive simulation. According to the simulation results, the proposed system can filter the text message and show high accuracy performance against the text message which cannot be filtered by the 3 major telecom companies.

Delay Fault Test Pattern Generator Using Indirect Implication Algorithms in Scan Environment (스캔 환경에서 간접 유추 알고리즘을 이용한 경로 지연 고장 검사 입력 생성기)

  • Kim, Won-Gi;Kim, Myeong-Gyun;Gang, Seong-Ho
    • The Transactions of the Korea Information Processing Society
    • /
    • v.6 no.6
    • /
    • pp.1656-1666
    • /
    • 1999
  • The more complex and large digital circuits become, the more important delay test becomes which guarantees that circuits operate in time. In this paper, the proposed algorithm is developed, which enable the fast indirect implication for efficient test pattern generation in sequential circuits of standard scan environment. Static learning algorithm enables application of a new implication value using contrapositive proposition. The static learning procedure found structurally, analyzes the gate structure in the preprocessing phase and store the information of learning occurrence so that it can be used in the test pattern generation procedure if it satisfies the implication condition. If there exists a signal line which include all paths from some particular primary inputs, it is a partitioning point. If paths passing that point have the same partial path from primary input to the signal or from the signal to primary output, they will need the same primary input values which separated by the partitioning point. In this paper test pattern generation can be more effective by using this partitioning technique. Finally, an efficient delay fault test pattern generator using indirect implication is developed and the effectiveness of these algorithms is demonstrated by experiments.

  • PDF

A Study on Data Preprocessing for the Activity-Travel Simulator: A Case of FEATHERS Seoul (활동기반 시뮬레이터 입력 자료의 전처리 방안에 대한 연구: FEATHERS Seoul을 사례로)

  • Cho, Sungjin;Hwang, Jeong Hwan;Bellemans, Tom;Kochan, Bruno;Lee, Won Do;Choi, Keechoo;Joh, Chang-Hyeon
    • Journal of Korean Society of Transportation
    • /
    • v.32 no.5
    • /
    • pp.531-543
    • /
    • 2014
  • Research on activity-based travel demand forecasting and activity-travel simulator has received an international attention for the last two decades. Ways to develop the activity-based simulator may be manifold. It is obvious that importing an existing simulator that has been proven internationally likely reduces the development cost and the risk of failure. By definition of the activity-based approach, however, the details of an activity-based simulator inevitably relies on particular social, economic and cultural characteristics of the society where the simulator is developed. When importing such a simulator from overseas, the researcher should be aware of the importance of tuning the system for the society to which the imported system is applied. There are many potential works on this, including for example the tuning of data structure that is likely different form of the original system. The authors are yet aware of certain research on those. The current paper aims to report the result of transforming the input data for applying the existing activity-travel simulator to Seoul. The paper first introduces FEATHERS that was developed in Belgium having Albatross which is the core of system. FEATHERS Seoul that is under development and modified version of the original FEATHERS is briefly described and the related problems are discussed. The paper then explored to resolve and to alleviate such problems.

An Object Detection and Tracking System using Fuzzy C-means and CONDENSATION (Fuzzy C-means와 CONDENSATION을 이용한 객체 검출 및 추적 시스템)

  • Kim, Jong-Ho;Kim, Sang-Kyoon;Hang, Goo-Seun;Ahn, Sang-Ho;Kang, Byoung-Doo
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.16 no.4
    • /
    • pp.87-98
    • /
    • 2011
  • Detecting a moving object from videos and tracking it are basic and necessary preprocessing steps in many video systems like object recognition, context aware, and intelligent visual surveillance. In this paper, we propose a method that is able to detect a moving object quickly and accurately in a condition that background and light change in a real time. Furthermore, our system detects strongly an object in a condition that the target object is covered with other objects. For effective detection, effective Eigen-space and FCM are combined and employed, and a CONDENSATION algorithm is used to trace a detected object strongly. First, training data collected from a background image are linear-transformed using Principal Component Analysis (PCA). Second, an Eigen-background is organized from selected principal components having excellent discrimination ability on an object and a background. Next, an object is detected with FCM that uses a convolution result of the Eigen-vector of previous steps and the input image. Finally, an object is tracked by using coordinates of an detected object as an input value of condensation algorithm. Images including various moving objects in a same time are collected and used as training data to realize our system that is able to be adapted to change of light and background in a fixed camera. The result of test shows that the proposed method detects an object strongly in a condition having a change of light and a background, and partial movement of an object.