• Title/Summary/Keyword: classifier evaluation

Search Result 149, Processing Time 0.026 seconds

Radiomics-based Machine Learning Approach for Quantitative Classification of Spinal Metastases in Computed Tomography (컴퓨터 단층 촬영 영상에서의 전이성 척추 종양의 정량적 분류를 위한 라디오믹스 기반의 머신러닝 기법)

  • Lee, Eun Woo;Lim, Sang Heon;Jeon, Ji Soo;Kang, Hye Won;Kim, Young Jae;Jeon, Ji Young;Kim, Kwang Gi
    • Journal of Biomedical Engineering Research
    • /
    • v.42 no.3
    • /
    • pp.71-79
    • /
    • 2021
  • Currently, the naked eyes-based diagnosis of bone metastases on CT images relies on qualitative assessment. For this reason, there is a great need for a state-of-the-art approach that can assess and follow-up the bone metastases with quantitative biomarker. Radiomics can be used as a biomarker for objective lesion assessment by extracting quantitative numerical values from digital medical images. In this study, therefore, we evaluated the clinical applicability of non-invasive and objective bone metastases computer-aided diagnosis using radiomics-based biomarkers in CT. We employed a total of 21 approaches consist of three-classifiers and seven-feature selection methods to predict bone metastases and select biomarkers. We extracted three-dimensional features from the CT that three groups consisted of osteoblastic, osteolytic, and normal-healthy vertebral bodies. For evaluation, we compared the prediction results of the classifiers with the medical staff's diagnosis results. As a result of the three-class-classification performance evaluation, we demonstrated that the combination of the random forest classifier and the sequential backward selection feature selection approach reached AUC of 0.74 on average. Moreover, we confirmed that 90-percentile, kurtosis, and energy were the features that contributed high in the classification of bone metastases in this approach. We expect that selected quantitative features will be helpful as biomarkers in improving the patient's survival and quality of life.

Refinement of damage identification capability of neural network techniques in application to a suspension bridge

  • Wang, J.Y.;Ni, Y.Q.
    • Structural Monitoring and Maintenance
    • /
    • v.2 no.1
    • /
    • pp.77-93
    • /
    • 2015
  • The idea of using measured dynamic characteristics for damage detection is attractive because it allows for a global evaluation of the structural health and condition. However, vibration-based damage detection for complex structures such as long-span cable-supported bridges still remains a challenge. As a suspension or cable-stayed bridge involves in general thousands of structural components, the conventional damage detection methods based on model updating and/or parameter identification might result in ill-conditioning and non-uniqueness in the solution of inverse problems. Alternatively, methods that utilize, to the utmost extent, information from forward problems and avoid direct solution to inverse problems would be more suitable for vibration-based damage detection of long-span cable-supported bridges. The auto-associative neural network (ANN) technique and the probabilistic neural network (PNN) technique, that both eschew inverse problems, have been proposed for identifying and locating damage in suspension and cable-stayed bridges. Without the help of a structural model, ANNs with appropriate configuration can be trained using only the measured modal frequencies from healthy structure under varying environmental conditions, and a new set of modal frequency data acquired from an unknown state of the structure is then fed into the trained ANNs for damage presence identification. With the help of a structural model, PNNs can be configured using the relative changes of modal frequencies before and after damage by assuming damage at different locations, and then the measured modal frequencies from the structure can be presented to locate the damage. However, such formulated ANNs and PNNs may still be incompetent to identify damage occurring at the deck members of a cable-supported bridge because of very low modal sensitivity to the damage. The present study endeavors to enhance the damage identification capability of ANNs and PNNs when being applied for identification of damage incurred at deck members. Effort is first made to construct combined modal parameters which are synthesized from measured modal frequencies and modal shape components to train ANNs for damage alarming. With the purpose of improving identification accuracy, effort is then made to configure PNNs for damage localization by adapting the smoothing parameter in the Bayesian classifier to different values for different pattern classes. The performance of the ANNs with their input being modal frequencies and the combined modal parameters respectively and the PNNs with constant and adaptive smoothing parameters respectively is evaluated through simulation studies of identifying damage inflicted on different deck members of the double-deck suspension Tsing Ma Bridge.

Statistical Analysis of Projection-Based Face Recognition Algorithms (투사에 기초한 얼굴 인식 알고리즘들의 통계적 분석)

  • 문현준;백순화;전병민
    • The Journal of Korean Institute of Communications and Information Sciences
    • /
    • v.25 no.5A
    • /
    • pp.717-725
    • /
    • 2000
  • Within the last several years, there has been a large number of algorithms developed for face recognition. The majority of these algorithms have been view- and projection-based algorithms. Our definition of projection is not restricted to projecting the image onto an orthogonal basis the definition is expansive and includes a general class of linear transformation of the image pixel values. The class includes correlation, principal component analysis, clustering, gray scale projection, and matching pursuit filters. In this paper, we perform a detailed analysis of this class of algorithms by evaluating them on the FERET database of facial images. In our experiments, a projection-based algorithms consists of three steps. The first step is done off-line and determines the new basis for the images. The bases is either set by the algorithm designer or is learned from a training set. The last two steps are on-line and perform the recognition. The second step projects an image onto the new basis and the third step recognizes a face in an with a nearest neighbor classifier. The classification is performed in the projection space. Most evaluation methods report algorithm performance on a single gallery. This does not fully capture algorithm performance. In our study, we construct set of independent galleries. This allows us to see how individual algorithm performance varies over different galleries. In addition, we report on the relative performance of the algorithms over the different galleries.

  • PDF

Design of Optimized RBFNNs based on Night Vision Face Recognition Simulator Using the 2D2 PCA Algorithm ((2D)2 PCA알고리즘을 이용한 최적 RBFNNs 기반 나이트비전 얼굴인식 시뮬레이터 설계)

  • Jang, Byoung-Hee;Kim, Hyun-Ki;Oh, Sung-Kwun
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.24 no.1
    • /
    • pp.1-6
    • /
    • 2014
  • In this study, we propose optimized RBFNNs based on night vision face recognition simulator with the aid of $(2D)^2$ PCA algorithm. It is difficult to obtain the night image for performing face recognition due to low brightness in case of image acquired through CCD camera at night. For this reason, a night vision camera is used to get images at night. Ada-Boost algorithm is also used for the detection of face images on both face and non-face image area. And the minimization of distortion phenomenon of the images is carried out by using the histogram equalization. These high-dimensional images are reduced to low-dimensional images by using $(2D)^2$ PCA algorithm. Face recognition is performed through polynomial-based RBFNNs classifier, and the essential design parameters of the classifiers are optimized by means of Differential Evolution(DE). The performance evaluation of the optimized RBFNNs based on $(2D)^2$ PCA is carried out with the aid of night vision face recognition system and IC&CI Lab data.

Big Data Management in Structured Storage Based on Fintech Models for IoMT using Machine Learning Techniques (기계학습법을 이용한 IoMT 핀테크 모델을 기반으로 한 구조화 스토리지에서의 빅데이터 관리 연구)

  • Kim, Kyung-Sil
    • Advanced Industrial SCIence
    • /
    • v.1 no.1
    • /
    • pp.7-15
    • /
    • 2022
  • To adopt the development in the medical scenario IoT developed towards the advancement with the processing of a large amount of medical data defined as an Internet of Medical Things (IoMT). The vast range of collected medical data is stored in the cloud in the structured manner to process the collected healthcare data. However, it is difficult to handle the huge volume of the healthcare data so it is necessary to develop an appropriate scheme for the healthcare structured data. In this paper, a machine learning mode for processing the structured heath care data collected from the IoMT is suggested. To process the vast range of healthcare data, this paper proposed an MTGPLSTM model for the processing of the medical data. The proposed model integrates the linear regression model for the processing of healthcare information. With the developed model outlier model is implemented based on the FinTech model for the evaluation and prediction of the COVID-19 healthcare dataset collected from the IoMT. The proposed MTGPLSTM model comprises of the regression model to predict and evaluate the planning scheme for the prevention of the infection spreading. The developed model performance is evaluated based on the consideration of the different classifiers such as LR, SVR, RFR, LSTM and the proposed MTGPLSTM model and the different size of data as 1GB, 2GB and 3GB is mainly concerned. The comparative analysis expressed that the proposed MTGPLSTM model achieves ~4% reduced MAPE and RMSE value for the worldwide data; in case of china minimal MAPE value of 0.97 is achieved which is ~ 6% minimal than the existing classifier leads.

Evaluation of Future Hydrologic Risk of Drought in Nakdong River Basin Using Bayesian Classification-Based Composite Drought Index (베이지안 분류 기반 통합가뭄지수를 활용한 낙동강 유역의 미래 가뭄에 대한 수문학적 위험도 분석)

  • Kim, Hyeok;Kim, Ji Eun;Kim, Jiyoung;Yoo, Jiyoung;Kim, Tae-Woong
    • KSCE Journal of Civil and Environmental Engineering Research
    • /
    • v.43 no.3
    • /
    • pp.309-319
    • /
    • 2023
  • Recently, the frequency and intensity of meteorological disasters have increased due to climate change. In South Korea, there are regional differences in vulnerability and response capability to cope with climate change because of regional climate characteristics. In particular, drought results from various factors and is linked to extensive meteorological, hydrological, and agricultural impacts. Therefore, in order to effectively cope with drought, it is necessary to use a composite drought index that can take into account various factors, and to evaluate future droughts comprehensively considering climate change. This study evaluated hydrologic risk(${\bar{R}}$) of future drought in the Nakdong River basin based on the Dynamic Naive Bayesian Classification (DNBC)-based composite drought index, which was calculated by applying Standardized Precipitation Index (SPI), Streamflow Drought Index (SDI), Evaporate Stress Index (ESI) and Water Supply Capacity Index (WSCI) to the DNBC. The indices used in the DNBC were calculated using observation data and climate scenario data. A bivariate frequency analysis was performed for the severity and duration of the composite drought. Then using the estimated bivariate return periods, hydrologic risks of drought were calculated for observation and future periods. The overall results indicated that there were the highest risks during the future period (2021-2040) (${\bar{R}}$=0.572), and Miryang River (#2021) had the highest risk (${\bar{R}}$=0.940) on average. The hydrologic risk of the Nakdong River basin will increase highly in the near future (2021-2040). During the far future (2041-2099), the hydrologic risk decreased in the northern basins, and increased in the southern basins.

Multi-Time Window Feature Extraction Technique for Anger Detection in Gait Data

  • Beom Kwon;Taegeun Oh
    • Journal of the Korea Society of Computer and Information
    • /
    • v.28 no.4
    • /
    • pp.41-51
    • /
    • 2023
  • In this paper, we propose a technique of multi-time window feature extraction for anger detection in gait data. In the previous gait-based emotion recognition methods, the pedestrian's stride, time taken for one stride, walking speed, and forward tilt angles of the neck and thorax are calculated. Then, minimum, mean, and maximum values are calculated for the entire interval to use them as features. However, each feature does not always change uniformly over the entire interval but sometimes changes locally. Therefore, we propose a multi-time window feature extraction technique that can extract both global and local features, from long-term to short-term. In addition, we also propose an ensemble model that consists of multiple classifiers. Each classifier is trained with features extracted from different multi-time windows. To verify the effectiveness of the proposed feature extraction technique and ensemble model, a public three-dimensional gait dataset was used. The simulation results demonstrate that the proposed ensemble model achieves the best performance compared to machine learning models trained with existing feature extraction techniques for four performance evaluation metrics.

A Deep Learning Based Approach to Recognizing Accompanying Status of Smartphone Users Using Multimodal Data (스마트폰 다종 데이터를 활용한 딥러닝 기반의 사용자 동행 상태 인식)

  • Kim, Kilho;Choi, Sangwoo;Chae, Moon-jung;Park, Heewoong;Lee, Jaehong;Park, Jonghun
    • Journal of Intelligence and Information Systems
    • /
    • v.25 no.1
    • /
    • pp.163-177
    • /
    • 2019
  • As smartphones are getting widely used, human activity recognition (HAR) tasks for recognizing personal activities of smartphone users with multimodal data have been actively studied recently. The research area is expanding from the recognition of the simple body movement of an individual user to the recognition of low-level behavior and high-level behavior. However, HAR tasks for recognizing interaction behavior with other people, such as whether the user is accompanying or communicating with someone else, have gotten less attention so far. And previous research for recognizing interaction behavior has usually depended on audio, Bluetooth, and Wi-Fi sensors, which are vulnerable to privacy issues and require much time to collect enough data. Whereas physical sensors including accelerometer, magnetic field and gyroscope sensors are less vulnerable to privacy issues and can collect a large amount of data within a short time. In this paper, a method for detecting accompanying status based on deep learning model by only using multimodal physical sensor data, such as an accelerometer, magnetic field and gyroscope, was proposed. The accompanying status was defined as a redefinition of a part of the user interaction behavior, including whether the user is accompanying with an acquaintance at a close distance and the user is actively communicating with the acquaintance. A framework based on convolutional neural networks (CNN) and long short-term memory (LSTM) recurrent networks for classifying accompanying and conversation was proposed. First, a data preprocessing method which consists of time synchronization of multimodal data from different physical sensors, data normalization and sequence data generation was introduced. We applied the nearest interpolation to synchronize the time of collected data from different sensors. Normalization was performed for each x, y, z axis value of the sensor data, and the sequence data was generated according to the sliding window method. Then, the sequence data became the input for CNN, where feature maps representing local dependencies of the original sequence are extracted. The CNN consisted of 3 convolutional layers and did not have a pooling layer to maintain the temporal information of the sequence data. Next, LSTM recurrent networks received the feature maps, learned long-term dependencies from them and extracted features. The LSTM recurrent networks consisted of two layers, each with 128 cells. Finally, the extracted features were used for classification by softmax classifier. The loss function of the model was cross entropy function and the weights of the model were randomly initialized on a normal distribution with an average of 0 and a standard deviation of 0.1. The model was trained using adaptive moment estimation (ADAM) optimization algorithm and the mini batch size was set to 128. We applied dropout to input values of the LSTM recurrent networks to prevent overfitting. The initial learning rate was set to 0.001, and it decreased exponentially by 0.99 at the end of each epoch training. An Android smartphone application was developed and released to collect data. We collected smartphone data for a total of 18 subjects. Using the data, the model classified accompanying and conversation by 98.74% and 98.83% accuracy each. Both the F1 score and accuracy of the model were higher than the F1 score and accuracy of the majority vote classifier, support vector machine, and deep recurrent neural network. In the future research, we will focus on more rigorous multimodal sensor data synchronization methods that minimize the time stamp differences. In addition, we will further study transfer learning method that enables transfer of trained models tailored to the training data to the evaluation data that follows a different distribution. It is expected that a model capable of exhibiting robust recognition performance against changes in data that is not considered in the model learning stage will be obtained.

The Effect of Meta-Features of Multiclass Datasets on the Performance of Classification Algorithms (다중 클래스 데이터셋의 메타특징이 판별 알고리즘의 성능에 미치는 영향 연구)

  • Kim, Jeonghun;Kim, Min Yong;Kwon, Ohbyung
    • Journal of Intelligence and Information Systems
    • /
    • v.26 no.1
    • /
    • pp.23-45
    • /
    • 2020
  • Big data is creating in a wide variety of fields such as medical care, manufacturing, logistics, sales site, SNS, and the dataset characteristics are also diverse. In order to secure the competitiveness of companies, it is necessary to improve decision-making capacity using a classification algorithm. However, most of them do not have sufficient knowledge on what kind of classification algorithm is appropriate for a specific problem area. In other words, determining which classification algorithm is appropriate depending on the characteristics of the dataset was has been a task that required expertise and effort. This is because the relationship between the characteristics of datasets (called meta-features) and the performance of classification algorithms has not been fully understood. Moreover, there has been little research on meta-features reflecting the characteristics of multi-class. Therefore, the purpose of this study is to empirically analyze whether meta-features of multi-class datasets have a significant effect on the performance of classification algorithms. In this study, meta-features of multi-class datasets were identified into two factors, (the data structure and the data complexity,) and seven representative meta-features were selected. Among those, we included the Herfindahl-Hirschman Index (HHI), originally a market concentration measurement index, in the meta-features to replace IR(Imbalanced Ratio). Also, we developed a new index called Reverse ReLU Silhouette Score into the meta-feature set. Among the UCI Machine Learning Repository data, six representative datasets (Balance Scale, PageBlocks, Car Evaluation, User Knowledge-Modeling, Wine Quality(red), Contraceptive Method Choice) were selected. The class of each dataset was classified by using the classification algorithms (KNN, Logistic Regression, Nave Bayes, Random Forest, and SVM) selected in the study. For each dataset, we applied 10-fold cross validation method. 10% to 100% oversampling method is applied for each fold and meta-features of the dataset is measured. The meta-features selected are HHI, Number of Classes, Number of Features, Entropy, Reverse ReLU Silhouette Score, Nonlinearity of Linear Classifier, Hub Score. F1-score was selected as the dependent variable. As a result, the results of this study showed that the six meta-features including Reverse ReLU Silhouette Score and HHI proposed in this study have a significant effect on the classification performance. (1) The meta-features HHI proposed in this study was significant in the classification performance. (2) The number of variables has a significant effect on the classification performance, unlike the number of classes, but it has a positive effect. (3) The number of classes has a negative effect on the performance of classification. (4) Entropy has a significant effect on the performance of classification. (5) The Reverse ReLU Silhouette Score also significantly affects the classification performance at a significant level of 0.01. (6) The nonlinearity of linear classifiers has a significant negative effect on classification performance. In addition, the results of the analysis by the classification algorithms were also consistent. In the regression analysis by classification algorithm, Naïve Bayes algorithm does not have a significant effect on the number of variables unlike other classification algorithms. This study has two theoretical contributions: (1) two new meta-features (HHI, Reverse ReLU Silhouette score) was proved to be significant. (2) The effects of data characteristics on the performance of classification were investigated using meta-features. The practical contribution points (1) can be utilized in the development of classification algorithm recommendation system according to the characteristics of datasets. (2) Many data scientists are often testing by adjusting the parameters of the algorithm to find the optimal algorithm for the situation because the characteristics of the data are different. In this process, excessive waste of resources occurs due to hardware, cost, time, and manpower. This study is expected to be useful for machine learning, data mining researchers, practitioners, and machine learning-based system developers. The composition of this study consists of introduction, related research, research model, experiment, conclusion and discussion.