• Title/Summary/Keyword: Unsupervised algorithm

Search Result 279, Processing Time 0.024 seconds

Anomaly Data Detection Using Machine Learning in Crowdsensing System (크라우드센싱 시스템에서 머신러닝을 이용한 이상데이터 탐지)

  • Kim, Mihui;Lee, Gihun
    • Journal of IKEEE
    • /
    • v.24 no.2
    • /
    • pp.475-485
    • /
    • 2020
  • Recently, a crowdsensing system that provides a new sensing service with real-time sensing data provided from a user's device including a sensor without installing a separate sensor has attracted attention. In the crowdsensing system, meaningless data may be provided due to a user's operation error or communication problem, or false data may be provided to obtain compensation. Therefore, the detection and removal of the abnormal data determines the quality of the crowdsensing service. The proposed methods in the past to detect these anomalies are not efficient for the fast-changing environment of crowdsensing. This paper proposes an anomaly data detection method by extracting the characteristics of continuously and rapidly changing sensing data environment by using machine learning technology and modeling it with an appropriate algorithm. We show the performance and feasibility of the proposed system using deep learning binary classification model of supervised learning and autoencoder model of unsupervised learning.

Comparison between Possibilistic c-Means (PCM) and Artificial Neural Network (ANN) Classification Algorithms in Land use/ Land cover Classification

  • Ganbold, Ganchimeg;Chasia, Stanley
    • International Journal of Knowledge Content Development & Technology
    • /
    • v.7 no.1
    • /
    • pp.57-78
    • /
    • 2017
  • There are several statistical classification algorithms available for land use/land cover classification. However, each has a certain bias or compromise. Some methods like the parallel piped approach in supervised classification, cannot classify continuous regions within a feature. On the other hand, while unsupervised classification method takes maximum advantage of spectral variability in an image, the maximally separable clusters in spectral space may not do much for our perception of important classes in a given study area. In this research, the output of an ANN algorithm was compared with the Possibilistic c-Means an improvement of the fuzzy c-Means on both moderate resolutions Landsat8 and a high resolution Formosat 2 images. The Formosat 2 image comes with an 8m spectral resolution on the multispectral data. This multispectral image data was resampled to 10m in order to maintain a uniform ratio of 1:3 against Landsat 8 image. Six classes were chosen for analysis including: Dense forest, eucalyptus, water, grassland, wheat and riverine sand. Using a standard false color composite (FCC), the six features reflected differently in the infrared region with wheat producing the brightest pixel values. Signature collection per class was therefore easily obtained for all classifications. The output of both ANN and FCM, were analyzed separately for accuracy and an error matrix generated to assess the quality and accuracy of the classification algorithms. When you compare the results of the two methods on a per-class-basis, ANN had a crisper output compared to PCM which yielded clusters with pixels especially on the moderate resolution Landsat 8 imagery.

Classification of Seoul Metro Stations Based on Boarding/ Alighting Patterns Using Machine Learning Clustering (기계학습 클러스터링을 이용한 승하차 패턴에 따른 서울시 지하철역 분류)

  • Min, Meekyung
    • The Journal of the Institute of Internet, Broadcasting and Communication
    • /
    • v.18 no.4
    • /
    • pp.13-18
    • /
    • 2018
  • In this study, we classify Seoul metro stations according to boarding and alighting patterns using machine earning technique. The target data is the number of boarding and alighting passengers per hour every day at 233 subway stations from 2008 to 2017 provided by the public data portal. Gaussian mixture model (GMM) and K-means clustering are used as machine learning techniques in order to classify subway stations. The distribution of the boarding time and the alighting time of the passengers can be modeled by the Gaussian mixture model. K-means clustering algorithm is used for unsupervised learning based on the data obtained by GMM modeling. As a result of the research, Seoul metro stations are classified into four groups according to boarding and alighting patterns. The results of this study can be utilized as a basic knowledge for analyzing the characteristics of Seoul subway stations and analyzing it economically, socially and culturally. The method of this research can be applied to public data and big data in areas requiring clustering.

High Resolution Satellite Image Segmentation Algorithm Development Using Seed-based region growing (시드 기반 영역확장기법을 이용한 고해상도 위성영상 분할기법 개발)

  • Byun, Young-Gi;Kim, Yong-Il
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.28 no.4
    • /
    • pp.421-430
    • /
    • 2010
  • Image segmentation technique is becoming increasingly important in the field of remote sensing image analysis in areas such as object oriented image classification to extract object regions of interest within images. This paper presents a new method for image segmentation in High Resolution Remote Sensing Image based on Improved Seeded Region Growing (ISRG) and Region merging. Firstly, multi-spectral edge detection was done using an entropy operator in pan-sharpened QuickBird imagery. Then, the initial seeds were automatically selected from the obtained multi-spectral edge map. After automatic selection of significant seeds, an initial segmentation was achieved by applying ISRG to consider spectral and edge information. Finally the region merging process, integrating region texture and spectral information, was carried out to get the final segmentation result. The accuracy assesment was done using the unsupervised objective evaluation method for evaluating the effectiveness of the proposed method. Experimental results demonstrated that the proposed method has good potential for application in the segmentation of high resolution satellite images.

A Design on Informal Big Data Topic Extraction System Based on Spark Framework (Spark 프레임워크 기반 비정형 빅데이터 토픽 추출 시스템 설계)

  • Park, Kiejin
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.5 no.11
    • /
    • pp.521-526
    • /
    • 2016
  • As on-line informal text data have massive in its volume and have unstructured characteristics in nature, there are limitations in applying traditional relational data model technologies for data storage and data analysis jobs. Moreover, using dynamically generating massive social data, social user's real-time reaction analysis tasks is hard to accomplish. In the paper, to capture easily the semantics of massive and informal on-line documents with unsupervised learning mechanism, we design and implement automatic topic extraction systems according to the mass of the words that consists a document. The input data set to the proposed system are generated first, using N-gram algorithm to build multiple words to capture the meaning of the sentences precisely, and Hadoop and Spark (In-memory distributed computing framework) are adopted to run topic model. In the experiment phases, TB level input data are processed for data preprocessing and proposed topic extraction steps are applied. We conclude that the proposed system shows good performance in extracting meaningful topics in time as the intermediate results come from main memories directly instead of an HDD reading.

3D Human Shape Deformation using Deep Learning (딥러닝을 이용한 3차원 사람모델형상 변형)

  • Kim, DaeHee;Hwang, Bon-Woo;Lee, SeungWook;Kwak, Sooyeong
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.25 no.2
    • /
    • pp.19-27
    • /
    • 2020
  • Recently, rapid and accurate 3D models creation is required in various applications using virtual reality and augmented reality technology. In this paper, we propose an on-site learning based shape deformation method which transforms the clothed 3D human model into the shape of an input point cloud. The proposed algorithm consists of two main parts: one is pre-learning and the other is on-site learning. Each learning consists of encoder, template transformation and decoder network. The proposed network is learned by unsupervised method, which uses the Chamfer distance between the input point cloud form and the template vertices as the loss function. By performing on-site learning on the input point clouds during the inference process, the high accuracy of the inference results can be obtained and presented through experiments.

Automated Training from Landsat Image for Classification of SPOT-5 and QuickBird Images

  • Kim, Yong-Min;Kim, Yong-Il;Park, Wan-Yong;Eo, Yang-Dam
    • Korean Journal of Remote Sensing
    • /
    • v.26 no.3
    • /
    • pp.317-324
    • /
    • 2010
  • In recent years, many automatic classification approaches have been employed. An automatic classification method can be effective, time-saving and can produce objective results due to the exclusion of operator intervention. This paper proposes a classification method based on automated training for high resolution multispectral images using ancillary data. Generally, it is problematic to automatically classify high resolution images using ancillary data, because of the scale difference between the high resolution image and the ancillary data. In order to overcome this problem, the proposed method utilizes the classification results of a Landsat image as a medium for automatic classification. For the classification of a Landsat image, a maximum likelihood classification is applied to the image, and the attributes of ancillary data are entered as the training data. In the case of a high resolution image, a K-means clustering algorithm, an unsupervised classification, was conducted and the result was compared to the classification results of the Landsat image. Subsequently, the training data of the high resolution image was automatically extracted using regular rules based on a RELATIONAL matrix that shows the relation between the two results. Finally, a high resolution image was classified and updated using the extracted training data. The proposed method was applied to QuickBird and SPOT-5 images of non-accessible areas. The result showed good performance in accuracy assessments. Therefore, we expect that the method can be effectively used to automatically construct thematic maps for non-accessible areas and update areas that do not have any attributes in geographic information system.

A Reconstruction of Classification for Iris Species Using Euclidean Distance Based on a Machine Learning (머신러닝 기반 유클리드 거리를 이용한 붓꽃 품종 분류 재구성)

  • Nam, Soo-Tai;Shin, Seong-Yoon;Jin, Chan-Yong
    • Journal of the Korea Institute of Information and Communication Engineering
    • /
    • v.24 no.2
    • /
    • pp.225-230
    • /
    • 2020
  • Machine learning is an algorithm which learns a computer based on the data so that the computer can identify the trend of the data and predict the output of new input data. Machine learning can be classified into supervised learning, unsupervised learning, and reinforcement learning. Supervised learning is a way of learning a machine with given label of data. In other words, a method of inferring a function of the system through a pair of data and a label is used to predict a result using a function inferred about new input data. If the predicted value is continuous, regression analysis is used. If the predicted value is discrete, it is used as a classification. A result of analysis, no. 8 (5, 3.4, setosa), 27 (5, 3.4, setosa), 41 (5, 3.5, setosa), 44 (5, 3.5, setosa) and 40 (5.1, 3.4, setosa) in Table 3 were classified as the most similar Iris flower. Therefore, theoretical practical are suggested.

Why Should I Ban You! : X-FDS (Explainable FDS) Model Based on Online Game Payment Log (X-FDS : 게임 결제 로그 기반 XAI적용 이상 거래탐지 모델 연구)

  • Lee, Young Hun;Kim, Huy Kang
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.32 no.1
    • /
    • pp.25-38
    • /
    • 2022
  • With the diversification of payment methods and games, related financial accidents are causing serious problems for users and game companies. Recently, game companies have introduced an Fraud Detection System (FDS) for game payment systems to prevent financial incident. However, FDS is ineffective and cannot provide major evidence based on judgment results, as it requires constant change of detection patterns. In this paper, we analyze abnormal transactions among payment log data of real game companies to generate related features. One of the unsupervised learning models, Autoencoder, was used to build a model to detect abnormal transactions, which resulted in over 85% accuracy. Using X-FDS (Explainable FDS) with XAI-SHAP, we could understand that the variables with the highest explanation for anomaly detection were the amount of transaction, transaction medium, and the age of users. Based on X-FDS, we derive an improved detection model with an accuracy of 94% was finally derived by fine-tuning the importance of features that adversely affect the proposed model.

Sustainable Smart City Building-energy Management Based on Reinforcement Learning and Sales of ESS Power

  • Dae-Kug Lee;Seok-Ho Yoon;Jae-Hyeok Kwak;Choong-Ho Cho;Dong-Hoon Lee
    • KSII Transactions on Internet and Information Systems (TIIS)
    • /
    • v.17 no.4
    • /
    • pp.1123-1146
    • /
    • 2023
  • In South Korea, there have been many studies on efficient building-energy management using renewable energy facilities in single zero-energy houses or buildings. However, such management was limited due to spatial and economic problems. To realize a smart zero-energy city, studying efficient energy integration for the entire city, not just for a single house or building, is necessary. Therefore, this study was conducted in the eco-friendly energy town of Chungbuk Innovation City. Chungbuk successfully realized energy independence by converging new and renewable energy facilities for the first time in South Korea. This study analyzes energy data collected from public buildings in that town every minute for a year. We propose a smart city building-energy management model based on the results that combine various renewable energy sources with grid power. Supervised learning can determine when it is best to sell surplus electricity, or unsupervised learning can be used if there is a particular pattern or rule for energy use. However, it is more appropriate to use reinforcement learning to maximize rewards in an environment with numerous variables that change every moment. Therefore, we propose a power distribution algorithm based on reinforcement learning that considers the sales of Energy Storage System power from surplus renewable energy. Finally, we confirm through economic analysis that a 10% saving is possible from this efficiency.