• Title/Summary/Keyword: supervised clustering

Search Result 112, Processing Time 0.027 seconds

Parallel Processing of K-means Clustering Algorithm for Unsupervised Classification of Large Satellite Imagery (대용량 위성영상의 무감독 분류를 위한 K-means 군집화 알고리즘의 병렬처리)

  • Han, Soohee
    • Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography
    • /
    • v.35 no.3
    • /
    • pp.187-194
    • /
    • 2017
  • The present study introduces a method to parallelize k-means clustering algorithm for fast unsupervised classification of large satellite imagery. Known as a representative algorithm for unsupervised classification, k-means clustering is usually applied to a preprocessing step before supervised classification, but can show the evident advantages of parallel processing due to its high computational intensity and less human intervention. Parallel processing codes are developed by using multi-threading based on OpenMP. In experiments, a PC of 8 multi-core integrated CPU is involved. A 7 band and 30m resolution image from LANDSAT 8 OLI and a 8 band and 10m resolution image from Sentinel-2A are tested. Parallel processing has shown 6 time faster speed than sequential processing when using 10 classes. To check the consistency of parallel and sequential processing, centers, numbers of classified pixels of classes, classified images are mutually compared, resulting in the same results. The present study is meaningful because it has proved that performance of large satellite processing can be significantly improved by using parallel processing. And it is also revealed that it easy to implement parallel processing by using multi-threading based on OpenMP but it should be carefully designed to control the occurrence of false sharing.

A Comparison of the Land Cover Data Sets over Asian Region: USGS, IGBP, and UMd (아시아 지역 지면피복자료 비교 연구: USGS, IGBP, 그리고 UMd)

  • Kang, Jeon-Ho;Suh, Myoung-Seok;Kwak, Chong-Heum
    • Atmosphere
    • /
    • v.17 no.2
    • /
    • pp.159-169
    • /
    • 2007
  • A comparison of the three land cover data sets (United States Geological Survey: USGS, International Geosphere Biosphere Programme: IGBP, and University of Maryland: UMd), derived from 1992-1993 Advanced Very High Resolution Radiometer(AVHRR) data sets, was performed over the Asian continent. Preprocesses such as the unification of map projection and land cover definition, were applied for the comparison of the three different land cover data sets. Overall, the agreement among the three land cover data sets was relatively high for the land covers which have a distinct phenology, such as urban, open shrubland, mixed forest, and bare ground (>45%). The ratios of triple agreement (TA), couple agreement (CA) and total disagreement (TD) among the three land cover data sets are 30.99%, 57.89% and 8.91%, respectively. The agreement ratio between USGS and IGBP is much greater (about 80%) than that (about 32%) between USGS and UMd (or IGBP and UMd). The main reasons for the relatively low agreement among the three land cover data sets are differences in 1) the number of land cover categories, 2) the basic input data sets used for the classification, 3) classification (or clustering) methodologies, and 4) level of preprocessing. The number of categories for the USGS, IGBP and UMd are 24, 17 and 14, respectively. USGS and IGBP used only the 12 monthly normalized difference vegetation index (NDVI), whereas UMd used the 12 monthly NDVI and other 29 auxiliary data derived from AVHRR 5 channels. USGS and IGBP used unsupervised clustering method, whereas UMd used the supervised technique, decision tree using the ground truth data derived from the high resolution Landsat data. The insufficient preprocessing in USGS and IGBP compared to the UMd resulted in the spatial discontinuity and misclassification.

Candidate First Moves for Solving Life-and-Death Problems in the Game of Go, using Kohonen Neural Network (코호넨 신경망을 이용 바둑 사활문제를 풀기 위한 후보 첫 수들)

  • Lee, Byung-Doo;Keum, Young-Wook
    • Journal of Korea Game Society
    • /
    • v.9 no.1
    • /
    • pp.105-114
    • /
    • 2009
  • In the game of Go, the life-and-death problem is a fundamental problem to be definitely overcome when implementing a computer Go program. To solve local Go problems such as life-and-death problems, an important consideration is how to tackle the game tree's huge branching factor and its depth. The basic idea of the experiment conducted in this article is that we modelled the human behavior to get the recognized first moves to kill the surrounded group. In the game of Go, similar life-and-death problems(patterns) often have similar solutions. To categorize similar patterns, we implemented Kohonen Neural Network(KNN) based clustering and found that the experimental result is promising and thus can compete with a pattern matching method, that uses supervised learning with a neural network, for solving life-and-death problems.

  • PDF

Impurity profiling and chemometric analysis of methamphetamine seizures in Korea

  • Shin, Dong Won;Ko, Beom Jun;Cheong, Jae Chul;Lee, Wonho;Kim, Suhkmann;Kim, Jin Young
    • Analytical Science and Technology
    • /
    • v.33 no.2
    • /
    • pp.98-107
    • /
    • 2020
  • Methamphetamine (MA) is currently the most abused illicit drug in Korea. MA is produced by chemical synthesis, and the final target drug that is produced contains small amounts of the precursor chemicals, intermediates, and by-products. To identify and quantify these trace compounds in MA seizures, a practical and feasible approach for conducting chromatographic fingerprinting with a suite of traditional chemometric methods and recently introduced machine learning approaches was examined. This was achieved using gas chromatography (GC) coupled with a flame ionization detector (FID) and mass spectrometry (MS). Following appropriate examination of all the peaks in 71 samples, 166 impurities were selected as the characteristic components. Unsupervised (principal component analysis (PCA), hierarchical cluster analysis (HCA), and K-means clustering) and supervised (partial least squares-discriminant analysis (PLS-DA), orthogonal partial least squares-discriminant analysis (OPLS-DA), support vector machines (SVM), and deep neural network (DNN) with Keras) chemometric techniques were employed for classifying the 71 MA seizures. The results of the PCA, HCA, K-means clustering, PLS-DA, OPLS-DA, SVM, and DNN methods for quality evaluation were in good agreement. However, the tested MA seizures possessed distinct features, such as chirality, cutting agents, and boiling points. The study indicated that the established qualitative and semi-quantitative methods will be practical and useful analytical tools for characterizing trace compounds in illicit MA seizures. Moreover, they will provide a statistical basis for identifying the synthesis route, sources of supply, trafficking routes, and connections between seizures, which will support drug law enforcement agencies in their effort to eliminate organized MA crime.

Unified Labeling and Fine-Grained Verification for Improving Ground-Truth of Malware Analysis (악성코드 분석의 Ground-Truth 향상을 위한 Unified Labeling과 Fine-Grained 검증)

  • Oh, Sang-Jin;Park, Leo-Hyun;Kwon, Tae-Kyoung
    • Journal of the Korea Institute of Information Security & Cryptology
    • /
    • v.29 no.3
    • /
    • pp.549-555
    • /
    • 2019
  • According to a recent report by anti-virus vendors, the number of new and modified malware increased exponentially. Therefore, malware analysis research using machine learning has been actively researched in order to replace passive analysis method which has low analysis speed. However, when using supervised learning based machine learning, many studies use low-reliability malware family name provided by the antivirus vendor as the label. In order to solve the problem of low-reliability of malware label, this paper introduces a new labeling technique, "Unified Labeling", and further verifies the malicious behavior similarity through the feature analysis of the fine-grained method. To verify this study, various clustering algorithms were used and compared with existing labeling techniques.

Classification of Forest Vegetation Zone over Southern Part of Korean Peninsula Using Geographic Information Systems (環境因子의 空間分析을 통한 南韓지역의 山林植生帶 구분/지리정보시스템(GIS)에 의한 접근)

  • Lee, Kyu-Sung;Byong-Chun Lee;Joon Hwan Shin
    • The Korean Journal of Ecology
    • /
    • v.19 no.5
    • /
    • pp.465-476
    • /
    • 1996
  • There are several environmental variables that may be influential to the spatial distribution of forest vegetation. To create a map of forest vegetation zone over southern part of Korean Peninsula, digital map layers were produced for each of environmental variables that include topography, geographic locations, and climate. In addition, an extensive set of field survey data was collected at relatively undisturbed forests and they were introduced into the GIS database with exact coordinates of survey sites. Preliminary statistical analysis on the survey data showed that the environmental variables were significantly different among the previously defined five forest vegetation zones. Classification of the six layers of digital map representing environmental variables was carried out by a supervised classifier using the training statistics from field survey data and by a clustering algorithm. Although the maps from two classifiers were somewhat different due to the classification procedure applied, they showed overall patterns of vertical and horizontal distribution of forest zones. considering the spatial contents of many ecological studies, GIS can be used as an important tool to manage and analyze spatial data. This study discusses more about the generation of digital map and the analysis procedure rather than the outcome map of forest vegetation zone.

  • PDF

MONITORING OF MOUNTAINOUS AREAS USING SIMULATED IMAGES TO KOMPSAT-II

  • Chang Eun-Mi;Shin Soo-Hyun
    • Proceedings of the KSRS Conference
    • /
    • 2005.10a
    • /
    • pp.653-655
    • /
    • 2005
  • More than 70 percent of terrestrial territory of Korea is mountainous areas where degradation becomes serious year by year due to illegal tombs, expanding golf courses and stone mine development. We elaborate the potential usage of high resolution image for the monitoring of the phenomena. We made the classification of tombs and the statistical radiometric characteristics of graves were identified from this project. The graves could be classified to 4 groups from the field survey. As compared with grouping data after clustering and discriminant analysis, the two results coincided with each other. Object-oriented classification algorithm for feature extraction was theoretically researched in this project. And we did a pilot project, which was performed with mixed methods. That is, the conventional methods such as unsupervised and supervised classification were mixed up with the new method for feature extraction, object-oriented classification method. This methodology showed about $60\%$ classification accuracy for extracting tombs from satellite imagery. The extraction of tombs' geographical coordinates and graves themselves from satellite image was performed in this project. The stone mines and golf courses are extracted by NDVI and GVI. The accuracy of classification was around 89 percent. The location accuracy showed extraction of tombs from one-meter resolution image is cheaper and quicker way than GPS method. Finally we interviewed local government officers and made analyses on the current situation of mountainous area management and potential usage of KOMPSAT-II images. Based on the requirement analysis, we developed software, which is to management and monitoring system for mountainous area for local government.

  • PDF

A Two-Stage Document Page Segmentation Method using Morphological Distance Map and RBF Network (거리 사상 함수 및 RBF 네트워크의 2단계 알고리즘을 적용한 서류 레이아웃 분할 방법)

  • Shin, Hyun-Kyung
    • Journal of KIISE:Software and Applications
    • /
    • v.35 no.9
    • /
    • pp.547-553
    • /
    • 2008
  • We propose a two-stage document layout segmentation method. At the first stage, as top-down segmentation, morphological distance map algorithm extracts a collection of rectangular regions from a given input image. This preliminary result from the first stage is employed as input parameters for the process of next stage. At the second stage, a machine-learning algorithm is adopted RBF network, one of neural networks based on statistical model, is selected. In order for constructing the hidden layer of RBF network, a data clustering technique bared on the self-organizing property of Kohonen network is utilized. We present a result showing that the supervised neural network, trained by 300 number of sample data, improves the preliminary results of the first stage.

DeepCleanNet: Training Deep Convolutional Neural Network with Extremely Noisy Labels

  • Olimov, Bekhzod;Kim, Jeonghong
    • Journal of Korea Multimedia Society
    • /
    • v.23 no.11
    • /
    • pp.1349-1360
    • /
    • 2020
  • In recent years, Convolutional Neural Networks (CNNs) have been successfully implemented in different tasks of computer vision. Since CNN models are the representatives of supervised learning algorithms, they demand large amount of data in order to train the classifiers. Thus, obtaining data with correct labels is imperative to attain the state-of-the-art performance of the CNN models. However, labelling datasets is quite tedious and expensive process, therefore real-life datasets often exhibit incorrect labels. Although the issue of poorly labelled datasets has been studied before, we have noticed that the methods are very complex and hard to reproduce. Therefore, in this research work, we propose Deep CleanNet - a considerably simple system that achieves competitive results when compared to the existing methods. We use K-means clustering algorithm for selecting data with correct labels and train the new dataset using a deep CNN model. The technique achieves competitive results in both training and validation stages. We conducted experiments using MNIST database of handwritten digits with 50% corrupted labels and achieved up to 10 and 20% increase in training and validation sets accuracy scores, respectively.

Acoustic Emission Source Characterization and Fracture Behavior of Finite-width Plate with a Circular Hole Defect using Artificial Neural Network (인공신경회로망을 이용한 원공결함을 갖는 유한 폭 판재의 음향방출 음원특성과 파괴거동에 관한 연구)

  • Rhee, Zhang-Kyu;Woo, Chang-Ki
    • Transactions of the Korean Society of Machine Tool Engineers
    • /
    • v.18 no.2
    • /
    • pp.170-177
    • /
    • 2009
  • The objective of this study is to evaluate an acoustic emission (AE) source characterization and fracture behavior of the SM45C steel by using back-propagation neural network (BPN). In previous research Ref. [8] about k-nearest neighbor classifier (k-NNC) continuity, we used K-means clustering method as an unsupervised learning method for obtaining multi-variate AE main data sets, such as AE counts, energy, amplitude, risetime, duration and counts to peak. Similarly, we applied k-NNC and BPN as a supervised learning method for obtaining multi-variate AE working data sets. According to the error of convergence for determinant criterion Wilk's ${\lambda}$, heuristic criteria D&B(Rij) and Tou values are discussed. As a result, in k-NNC before fracture signal is detected or when fracture signal is detected, showed that produce some empty classes in BPN. And we confirmed that could save trouble in AE signal processing if suitable error of convergence or acceptable encoding error give to BPN.