• Title/Summary/Keyword: 오버 샘플링

Search Result 46, Processing Time 0.042 seconds

Handling Method of Imbalance Data for Machine Learning : Focused on Sampling (머신러닝을 위한 불균형 데이터 처리 방법 : 샘플링을 위주로)

  • Lee, Kyunam;Lim, Jongtae;Bok, Kyoungsoo;Yoo, Jaesoo
    • The Journal of the Korea Contents Association
    • /
    • v.19 no.11
    • /
    • pp.567-577
    • /
    • 2019
  • Recently, more and more attempts have been made to solve the problems faced by academia and industry through machine learning. Accordingly, various attempts are being made to solve non-general situations through machine learning, such as deviance, fraud detection and disability detection. A variety of attempts have been made to resolve the non-normal situation in which data is distributed disproportionately, generally resulting in errors. In this paper, we propose handling method of imbalance data for machine learning. The proposed method to such problem of an imbalance in data by verifying that the population distribution of major class is well extracted. Performance Evaluations have proven the proposed method to be better than the existing methods.

수치지도를 이용한 EOC영상의 반자동 기하보정

  • 안석범;박찬용;최준수;한광수;김천
    • Proceedings of the Korean Association of Geographic Inforamtion Studies Conference
    • /
    • 2003.04a
    • /
    • pp.575-580
    • /
    • 2003
  • KOMPSAT-1 위성의 EOC영상은 위성에서 지구를 촬영하는 동안 발생하는 영상 왜곡을 포함하고 있다. 본 연구는 EOC영상의 영상왜곡을 보정하기 위하여 수치지도를 이용하는 정밀기하보정에 대하여 연구한다. 정밀기하보정 과정은 수치지도와 EOC영상의 좌표계를 통합하는 과정을 거쳐 오버레이를 만들어 수치지도의 삼각점을 기준으로 위성영상에서 GCP를 선택하고, 이 GCP를 이용하여 위성 영상을 딜로니 삼각형들의 Mesh형태로 변환하여 모든 딜로니 삼각형을 리샘플링하는 과정을 거쳐 보정된 EOC영상을 얻는다.

  • PDF

Data Statical Analysis based Data Filtering Scheme for Monitoring System on Wireless Sensor Network (무선 센서 네트워크 모니터링 시스템을 위한 데이터 통계 분석 기반 데이터 필터링 기법)

  • Lee, Hyun-Jo;Choi, Young-Ho;Chang, Jae-Woo
    • The Journal of the Korea Contents Association
    • /
    • v.10 no.3
    • /
    • pp.53-63
    • /
    • 2010
  • Recently, various monitoring systems are implemented actively by using wireless sensor networks(WSN). When implementing WSN-based monitoring system, there are three important issues to consider. At First, we need to consider a sensor node failure detection method to support the ongoing monitoring. Secondly, because sensor nodes use limited battery power, we need an efficient data filtering method to reduce energy consumption. At Last, a reducing processing overhead method is necessary. The existing Kalman filtering scheme has good performance on data filtering, but it causes too much processing overhead to estimate sensed data. To solve these problems, we, in this paper, propose a new data filtering scheme based on data statical analysis. First, the proposed scheme periodically aggregates node survival massages to support a node failure detection. Secondly, to reduce energy consumption, it sends the sample data with a node survival massage and do data filtering based on those messages. Finally, it analyzes the sample data to estimate filtering range in a server. As a result, each sensor node can use only simple compare operation for filtering data. In addition, we show from our performance analysis that the proposed scheme outperforms the Kalman filtering scheme in terms of the number of sending messages.

Novel Polar Transmitter with 2-Bit Sigma-Delta Modulation (2비트 시그마-델타 변조를 이용한 새로운 폴라 트랜스미터)

  • Lim, Ji-Youn;Cheon, Sang-Hoon;Kim, Kyeong-Hak;Hong, Song-Cheol;Kim, Dong-Wook
    • The Journal of Korean Institute of Electromagnetic Engineering and Science
    • /
    • v.18 no.8
    • /
    • pp.970-976
    • /
    • 2007
  • This paper presents a novel polar transmitter architecture with a 2-bit sigma-delta modulator. In the proposed architecture, the 2-bit sigma-delta modulator is introduced to suppress quantization noise of conventional sigma-delta modulator. The power amplifier configuration is also modified in a binary form to accommodate the 2-bit digitized envelope signal. The Ptolemy simulation results of the proposed structure show that the spectral property is greatly improved in full transmit band of EDGE system. The fine quantization scheme of the 2-bit modulator lowers the noise level by 10dB without increasing the over-sampling ratio, which may be obtained if the over-sampling ratio increases twofold. Dynamic range is also enhanced up to 5dB owing to the new form of the power amplifier in the transmitter.

Study on Fault Detection of a Gas Pressure Regulator Based on Machine Learning Algorithms

  • Seo, Chan-Yang;Suh, Young-Joo;Kim, Dong-Ju
    • Journal of the Korea Society of Computer and Information
    • /
    • v.25 no.4
    • /
    • pp.19-27
    • /
    • 2020
  • In this paper, we propose a machine learning method for diagnosing the failure of a gas pressure regulator. Originally, when implementing a machine learning model for detecting abnormal operation of a facility, it is common to install sensors to collect data. However, failure of a gas pressure regulator can lead to fatal safety problems, so that installing an additional sensor on a gas pressure regulator is not simple. In this paper, we propose various machine learning approach for diagnosing the abnormal operation of a gas pressure regulator with only the flow rate and gas pressure data collected from a gas pressure regulator itself. Since the fault data of a gas pressure regulator is not enough, the model is trained in all classes by applying the over-sampling method. The classification model was implemented using Gradient boosting, 1D Convolutional Neural Networks, and LSTM algorithm, and gradient boosting model showed the best performance among classification models with 99.975% accuracy.

Mitigation of Impulse Noise Using Slew Rate Limiter in Oversampled Signal for Power Line Communication (전력선 통신에서 오버 샘플링과 Slew Rate 제한을 이용한 임펄스 잡음 제거 기법)

  • Oh, Woojin;Natarajan, Bala
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.23 no.4
    • /
    • pp.431-437
    • /
    • 2019
  • PLC(Power Line Communication) is being used in various ways in smart grid system because of the advantages of low cost and high data throughput. However, power line channel has many problems due to impulse noise and various studies have been conducted to solve the problem. Recently, ACDL(Adaptive Cannonical Differential Limiter) which is based on an adaptive clipping with analog nonlinear filter, has been proposed and performs better than the others. In this paper, we show that ACDL is similar to the detection of slew rate with oversampled digital signal by simplification and analysis. Through the simulation under the PRIME standard it is shown that the proposed performs equal to or better than that of ACDL, but significantly reduce the complexity to implement. The BER performance is equal but the complexity is reduced to less than 10%.

Proposal of Augmented Drought Inflow to Search Reliable Operational Policies for Water Supply Infrastructures (물 공급 시설의 신뢰성 있는 운영 계획 수립을 위한 가뭄 유입량 증강 기법의 제안)

  • Ji, Sukwang;Ahn, Kuk-Hyun
    • Proceedings of the Korea Water Resources Association Conference
    • /
    • 2022.05a
    • /
    • pp.189-189
    • /
    • 2022
  • 물 공급 시설의 효율적이고 안정적인 운영을 위한 운영 계획의 수립 및 검증을 위해서는 장기간의 유입량 자료가 필요하다. 하지만, 현실적으로 얻을 수 있는 실측 자료는 제한적이며, 유입량이 부족하여 댐 운영에 영향을 미치는 자료는 더욱 적을 수밖에 없다. 이를 개선하고자 장기간의 모의 유입량을 생성해 운영 계획을 수립하는 방법이 종종 사용되지만, 실측 자료를 기반으로 모의하기 때문에 이 역시 가뭄의 빈도가 낮아, 장기 가뭄이나 짧은 간격으로 가뭄이 발생할 시 안정적인 운영이 어렵다. 본 연구에서는 장기 가뭄 발생 시에도 안정적인 물 공급이 가능한 운영 계획 수립을 위해 가뭄 빈도를 증가시킨 유입량 모의 기법을 제안하고자 한다. 제안하는 모의 기법은 최근 머신러닝에서 사용되는 SMOTE 알고리즘을 기반으로 한다. SMOTE 알고리즘은 데이터의 불균형을 처리하기 위한 오버 샘플링 기법으로, 소수 그룹을 단순 복제하지 않고 새로운 복제본을 생성해 과적합의 위험이 적으며, 원자료의 정보가 손실되지 않는 장점이 있다. 본 연구에서는 미국 캘리포니아주에 위치한 Folsom 댐을 대상으로 고빈도 가뭄 유입량을 모의했으며, 고빈도 가뭄 유입량을 사용한 운영 계획을 수립하였다. Folsom 댐의 과거 관측 유입량 자료를 기반으로 고빈도 가뭄 유입량을 사용한 운영 계획과 일반적인 가뭄 빈도의 유입량을 사용한 운영 계획을 적용했을 때 발생하는 공급 부족량과 과잉 방류량의 차이를 비교해 고빈도 가뭄 유입량의 사용이 물 공급 시설의 안정적인 운영에 끼치는 영향을 확인하고자 한다.

  • PDF

Cross-Layer Handover Scheme Using Linear Regression Analysis in Mobile WiMAX Networks (선형 회귀 분석을 이용한 모바일 와이맥스에서 계층 통합적 핸드오버 기법)

  • Choi, Yong-Hoon;Yun, Seok-Yeul;Chung, Young-Uk;Kim, Beom-Joon;Lee, Jung-Ryun;Lee, Hyun-Joon
    • The Journal of The Korea Institute of Intelligent Transport Systems
    • /
    • v.8 no.2
    • /
    • pp.91-99
    • /
    • 2009
  • Mobile WiMAX is an emerging technology that can provide ubiquitous Internet access. To provide seamless service in mobile WiMAX environment, delay or disruption in dealing with mobility must be minimized. However offering seamless services on IEEE 802.16e networks is very hard due to long handover latency both in layer 2 and 3. In this paper, we propose a fast cross-layer handover scheme based on prediction algorithm. With the help of the prediction, layer-3 handover activities are able to occur prior to layer-2 handover, and therefore, total handover latency can be reduced. The experiments conducted with system parameters and propagation model defined by WiMAX Forum demonstrate that the proposed method predicts the future signal level accurately and reduces the total handover latency.

  • PDF

A New FeedForward(FF) Timing Estimation Technique for High-Speed Transmission of Bursts (고속의 버스트 전송을 위한 새로운 피드포워드 타이밍 추정 기법)

  • 최윤석;조지훈;김응배;차균현
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.25 no.12A
    • /
    • pp.1774-1780
    • /
    • 2000
  • 본 논문에서는 TDMA 방식의 고속의 버스트 데이터 전송에서 프리앰블의 오버샘플링 데이터 값을 이용한 새로운 피드포워드 타이밍 추정 기법을 제안한다. 제안된 추정 기법은 검출 오류 분산 값 (DEV : Detection Error Variance) 측면에서 기존의 여러 타이밍 추정기법과 MCRB (Modified Cramer-Rao Bound)와 비교되어 진다. 또한, 제안된 타이밍 추정 기법을 고정 샘플링 클럭과 타이밍 보정기로서 보간 필터를 이용한 심볼 동기 블록을 적용하여 이상적인 경우의 BER과 그 성능을 비교한 결과 이상적인 경우에 비해 성능 저하가 BER이 $10^{-3}$인 지점에서 최대 0.2dB 이내임을 확인하였다.

  • PDF

Conditional Generative Adversarial Network based Collaborative Filtering Recommendation System (Conditional Generative Adversarial Network(CGAN) 기반 협업 필터링 추천 시스템)

  • Kang, Soyi;Shin, Kyung-shik
    • Journal of Intelligence and Information Systems
    • /
    • v.27 no.3
    • /
    • pp.157-173
    • /
    • 2021
  • With the development of information technology, the amount of available information increases daily. However, having access to so much information makes it difficult for users to easily find the information they seek. Users want a visualized system that reduces information retrieval and learning time, saving them from personally reading and judging all available information. As a result, recommendation systems are an increasingly important technologies that are essential to the business. Collaborative filtering is used in various fields with excellent performance because recommendations are made based on similar user interests and preferences. However, limitations do exist. Sparsity occurs when user-item preference information is insufficient, and is the main limitation of collaborative filtering. The evaluation value of the user item matrix may be distorted by the data depending on the popularity of the product, or there may be new users who have not yet evaluated the value. The lack of historical data to identify consumer preferences is referred to as data sparsity, and various methods have been studied to address these problems. However, most attempts to solve the sparsity problem are not optimal because they can only be applied when additional data such as users' personal information, social networks, or characteristics of items are included. Another problem is that real-world score data are mostly biased to high scores, resulting in severe imbalances. One cause of this imbalance distribution is the purchasing bias, in which only users with high product ratings purchase products, so those with low ratings are less likely to purchase products and thus do not leave negative product reviews. Due to these characteristics, unlike most users' actual preferences, reviews by users who purchase products are more likely to be positive. Therefore, the actual rating data is over-learned in many classes with high incidence due to its biased characteristics, distorting the market. Applying collaborative filtering to these imbalanced data leads to poor recommendation performance due to excessive learning of biased classes. Traditional oversampling techniques to address this problem are likely to cause overfitting because they repeat the same data, which acts as noise in learning, reducing recommendation performance. In addition, pre-processing methods for most existing data imbalance problems are designed and used for binary classes. Binary class imbalance techniques are difficult to apply to multi-class problems because they cannot model multi-class problems, such as objects at cross-class boundaries or objects overlapping multiple classes. To solve this problem, research has been conducted to convert and apply multi-class problems to binary class problems. However, simplification of multi-class problems can cause potential classification errors when combined with the results of classifiers learned from other sub-problems, resulting in loss of important information about relationships beyond the selected items. Therefore, it is necessary to develop more effective methods to address multi-class imbalance problems. We propose a collaborative filtering model using CGAN to generate realistic virtual data to populate the empty user-item matrix. Conditional vector y identify distributions for minority classes and generate data reflecting their characteristics. Collaborative filtering then maximizes the performance of the recommendation system via hyperparameter tuning. This process should improve the accuracy of the model by addressing the sparsity problem of collaborative filtering implementations while mitigating data imbalances arising from real data. Our model has superior recommendation performance over existing oversampling techniques and existing real-world data with data sparsity. SMOTE, Borderline SMOTE, SVM-SMOTE, ADASYN, and GAN were used as comparative models and we demonstrate the highest prediction accuracy on the RMSE and MAE evaluation scales. Through this study, oversampling based on deep learning will be able to further refine the performance of recommendation systems using actual data and be used to build business recommendation systems.